`ultralytics 8.0.164` new StreamLoader `stream_buffer` argument (#4596)

Co-authored-by: jgoo9410 <jjoohhnnggooddwwiinn@gmail.com>
Co-authored-by: John Goodwin <johnf4g@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

.github/workflows/docker.yaml

@@ -118,7 +118,9 @@ jobs:
         run: |
           docker push ultralytics/ultralytics:${{ matrix.tags }}
           if [[ "${{ matrix.tags }}" == "latest" ]]; then
-            t=ultralytics/ultralytics:latest-runner && sudo docker build -f docker/Dockerfile-runner -t $t . && sudo docker push $t
+            t=ultralytics/ultralytics:latest-runner
+            docker build -f docker/Dockerfile-runner -t $t .
+            docker push $t
           fi
 
       - name: Notify on failure

docs/modes/predict.md

@@ -55,7 +55,7 @@ YOLOv8 can process different types of input sources for inference, as shown in t
     Use `stream=True` for processing long videos or large datasets to efficiently manage memory. When `stream=False`, the results for all frames or data points are stored in memory, which can quickly add up and cause out-of-memory errors for large inputs. In contrast, `stream=True` utilizes a generator, which only keeps the results of the current frame or data point in memory, significantly reducing memory consumption and preventing out-of-memory issues.
 
 | Source         | Argument                                   | Type            | Notes                                                                                       |
-|---------------|--------------------------------------------|-----------------|---------------------------------------------------------------------------------------------|
+|----------------|--------------------------------------------|-----------------|---------------------------------------------------------------------------------------------|
 | image          | `'image.jpg'`                              | `str` or `Path` | Single image file.                                                                          |
 | URL            | `'https://ultralytics.com/images/bus.jpg'` | `str`           | URL to an image.                                                                            |
 | screenshot     | `'screen'`                                 | `str`           | Capture a screenshot.                                                                       |
@@ -300,7 +300,7 @@ Below are code examples for using each source type:
 All supported arguments:
 
 | Name            | Type           | Default                | Description                                                                    |
-|----------------|----------------|------------------------|--------------------------------------------------------------------------------|
+|-----------------|----------------|------------------------|--------------------------------------------------------------------------------|
 | `source`        | `str`          | `'ultralytics/assets'` | source directory for images or videos                                          |
 | `conf`          | `float`        | `0.25`                 | object confidence threshold for detection                                      |
 | `iou`           | `float`        | `0.7`                  | intersection over union (IoU) threshold for NMS                                |
@@ -316,6 +316,7 @@ All supported arguments:
 | `hide_conf`     | `bool`         | `False`                | hide confidence scores                                                         |
 | `max_det`       | `int`          | `300`                  | maximum number of detections per image                                         |
 | `vid_stride`    | `bool`         | `False`                | video frame-rate stride                                                        |
+| `stream_buffer` | `bool`         | `False`                | buffer all streaming frames (True) or return the most recent frame (False)     |
 | `line_width`    | `None or int`  | `None`                 | The line width of the bounding boxes. If None, it is scaled to the image size. |
 | `visualize`     | `bool`         | `False`                | visualize model features                                                       |
 | `augment`       | `bool`         | `False`                | apply image augmentation to prediction sources                                 |

docs/usage/cfg.md

@@ -134,7 +134,7 @@ The training settings for YOLO models encompass various hyperparameters and conf
 The prediction settings for YOLO models encompass a range of hyperparameters and configurations that influence the model's performance, speed, and accuracy during inference on new data. Careful tuning and experimentation with these settings are essential to achieve optimal performance for a specific task. Key settings include the confidence threshold, Non-Maximum Suppression (NMS) threshold, and the number of classes considered. Additional factors affecting the prediction process are input data size and format, the presence of supplementary features such as masks or multiple labels per box, and the particular task the model is employed for.
 
 | Key             | Value                  | Description                                                                    |
-|----------------|------------------------|--------------------------------------------------------------------------------|
+|-----------------|------------------------|--------------------------------------------------------------------------------|
 | `source`        | `'ultralytics/assets'` | source directory for images or videos                                          |
 | `conf`          | `0.25`                 | object confidence threshold for detection                                      |
 | `iou`           | `0.7`                  | intersection over union (IoU) threshold for NMS                                |
@@ -149,6 +149,7 @@ The prediction settings for YOLO models encompass a range of hyperparameters and
 | `show_conf`     | `True`                 | show object confidence scores in plots                                         |
 | `max_det`       | `300`                  | maximum number of detections per image                                         |
 | `vid_stride`    | `False`                | video frame-rate stride                                                        |
+| `stream_buffer` | `bool`                 | buffer all streaming frames (True) or return the most recent frame (False)     |
 | `line_width`    | `None`                 | The line width of the bounding boxes. If None, it is scaled to the image size. |
 | `visualize`     | `False`                | visualize model features                                                       |
 | `augment`       | `False`                | apply image augmentation to prediction sources                                 |

ultralytics/__init__.py

@@ -1,6 +1,6 @@
 # Ultralytics YOLO 🚀, AGPL-3.0 license
 
-__version__ = '8.0.163'
+__version__ = '8.0.164'
 
 from ultralytics.models import RTDETR, SAM, YOLO
 from ultralytics.models.fastsam import FastSAM

ultralytics/cfg/default.yaml

@@ -60,6 +60,7 @@ save_crop: False  # (bool) save cropped images with results
 show_labels: True  # (bool) show object labels in plots
 show_conf: True  # (bool) show object confidence scores in plots
 vid_stride: 1  # (int) video frame-rate stride
+stream_buffer: False  # (bool) buffer all streaming frames (True) or return the most recent frame (False)
 line_width:   # (int, optional) line width of the bounding boxes, auto if missing
 visualize: False  # (bool) visualize model features
 augment: False  # (bool) apply image augmentation to prediction sources

ultralytics/data/build.py

@@ -135,7 +135,7 @@ def check_source(source):
     return source, webcam, screenshot, from_img, in_memory, tensor
 
 
-def load_inference_source(source=None, imgsz=640, vid_stride=1):
+def load_inference_source(source=None, imgsz=640, vid_stride=1, stream_buffer=False):
     """
     Loads an inference source for object detection and applies necessary transformations.
 
@@ -143,6 +143,7 @@ def load_inference_source(source=None, imgsz=640, vid_stride=1):
         source (str, Path, Tensor, PIL.Image, np.ndarray): The input source for inference.
         imgsz (int, optional): The size of the image for inference. Default is 640.
         vid_stride (int, optional): The frame interval for video sources. Default is 1.
+        stream_buffer (bool, optional): Determined whether stream frames will be buffered. Default is False.
 
     Returns:
         dataset (Dataset): A dataset object for the specified input source.
@@ -156,7 +157,7 @@ def load_inference_source(source=None, imgsz=640, vid_stride=1):
     elif in_memory:
         dataset = source
     elif webcam:
-        dataset = LoadStreams(source, imgsz=imgsz, vid_stride=vid_stride)
+        dataset = LoadStreams(source, imgsz=imgsz, vid_stride=vid_stride, stream_buffer=stream_buffer)
     elif screenshot:
         dataset = LoadScreenshots(source, imgsz=imgsz)
     elif from_img:

ultralytics/data/loaders.py

@@ -31,9 +31,10 @@ class SourceTypes:
 class LoadStreams:
     """YOLOv8 streamloader, i.e. `yolo predict source='rtsp://example.com/media.mp4'  # RTSP, RTMP, HTTP streams`."""
 
-    def __init__(self, sources='file.streams', imgsz=640, vid_stride=1):
+    def __init__(self, sources='file.streams', imgsz=640, vid_stride=1, stream_buffer=False):
         """Initialize instance variables and check for consistent input stream shapes."""
         torch.backends.cudnn.benchmark = True  # faster for fixed-size inference
+        self.stream_buffer = stream_buffer  # buffer input streams
         self.running = True  # running flag for Thread
         self.mode = 'stream'
         self.imgsz = imgsz
@@ -81,7 +82,7 @@ class LoadStreams:
         n, f = 0, self.frames[i]  # frame number, frame array
         while self.running and cap.isOpened() and n < (f - 1):
             # Only read a new frame if the buffer is empty
-            if not self.imgs[i]:
+            if not self.imgs[i] or not self.stream_buffer:
                 n += 1
                 cap.grab()  # .read() = .grab() followed by .retrieve()
                 if n % self.vid_stride == 0:
@@ -124,7 +125,16 @@ class LoadStreams:
             time.sleep(1 / min(self.fps))
 
         # Get and remove the next frame from imgs buffer
-        return self.sources, [x.pop(0) for x in self.imgs], None, ''
+        if self.stream_buffer:
+            images = [x.pop(0) for x in self.imgs]
+        else:
+            # Get the latest frame, and clear the rest from the imgs buffer
+            images = []
+            for x in self.imgs:
+                images.append(x.pop(-1) if x else None)
+                x.clear()
+
+        return self.sources, images, None, ''
 
     def __len__(self):
         """Return the length of the sources object."""

ultralytics/engine/predictor.py

@@ -209,7 +209,10 @@ class BasePredictor:
         self.imgsz = check_imgsz(self.args.imgsz, stride=self.model.stride, min_dim=2)  # check image size
         self.transforms = getattr(self.model.model, 'transforms', classify_transforms(
             self.imgsz[0])) if self.args.task == 'classify' else None
-        self.dataset = load_inference_source(source=source, imgsz=self.imgsz, vid_stride=self.args.vid_stride)
+        self.dataset = load_inference_source(source=source,
+                                             imgsz=self.imgsz,
+                                             vid_stride=self.args.vid_stride,
+                                             stream_buffer=self.args.stream_buffer)
         self.source_type = self.dataset.source_type
         if not getattr(self, 'stream', True) and (self.dataset.mode == 'stream' or  # streams
                                                   len(self.dataset) > 1000 or  # images