* Using Rayon to accelarate YOLO post-processing

* Refactor YOLO with outputs format

* Optimize `conf * clss` for yolov5 v6 v7

* Add depth-anything-v2

* Update README.md

* Update CHANGELOG.md
This commit is contained in:
Jamjamjon
2024-07-12 19:46:48 +08:00
committed by GitHub
parent 25d9088e2e
commit edc3a8897c
69 changed files with 1563 additions and 1203 deletions

181
examples/yolo/README.md Normal file
View File

@ -0,0 +1,181 @@
<h1 align='center'>YOLO-Series</h1>
| Detection | Instance Segmentation | Pose |
| :---------------: | :------------------------: |:---------------: |
| <img src='./demos/det.png' width="300px"> | <img src='./demos/seg.png' width="300px"> |<img src='./demos/pose.png' width="300px"> |
| Classification | Obb |
| :------------------------: |:------------------------: |
|<img src='./demos/cls.png' width="300px"> |<img src='./demos/obb-2.png' width="628px">
| Head Detection | Fall Detection | Trash Detection |
| :------------------------: |:------------------------: |:------------------------: |
|<img src='./demos/head.png' width="300px"> |<img src='./demos/falldown.png' width="300px">|<img src='./demos/trash.png' width="300px">
| YOLO-World | Face Parsing | FastSAM |
| :------------------------: |:------------------------: |:------------------------: |
|<img src='./demos/yolov8-world.png' width="300px"> |<img src='./demos/face-parsing.png' width="300px">|<img src='./demos/fastsam.png' width="300px">
## Quick Start
```Shell
# Classify
cargo run -r --example yolo -- --task classify --version v5 # YOLOv5
cargo run -r --example yolo -- --task classify --version v8 # YOLOv8
# Detect
cargo run -r --example yolo -- --task detect --version v5 # YOLOv5
cargo run -r --example yolo -- --task detect --version v6 # YOLOv6
cargo run -r --example yolo -- --task detect --version v7 # YOLOv7
cargo run -r --example yolo -- --task detect --version v8 # YOLOv8
cargo run -r --example yolo -- --task detect --version v9 # YOLOv9
cargo run -r --example yolo -- --task detect --version v10 # YOLOv10
cargo run -r --example yolo -- --task detect --version rtdetr # YOLOv8-RTDETR
cargo run -r --example yolo -- --task detect --version v8 --model yolov8s-world-v2-shoes.onnx # YOLOv8-world
# Pose
cargo run -r --example yolo -- --task pose --version v8 # YOLOv8-Pose
# Segment
cargo run -r --example yolo -- --task segment --version v5 # YOLOv5-Segment
cargo run -r --example yolo -- --task segment --version v8 # YOLOv8-Segment
cargo run -r --example yolo -- --task segment --version v8 --model FastSAM-s-dyn-f16.onnx # FastSAM
# Obb
cargo run -r --example yolo -- --task obb --version v8 # YOLOv8-Obb
```
<details close>
<summary>other options</summary>
`--source` to specify the input images
`--model` to specify the ONNX model
`--width --height` to specify the input resolution
`--nc` to specify the number of model's classes
`--plot` to annotate with inference results
`--profile` to profile
`--cuda --trt --coreml --device_id` to select device
`--half` to use float16 when using TensorRT EP
</details>
## YOLOs configs with `Options`
<details open>
<summary>Use official YOLO Models</summary>
```Rust
let options = Options::default()
.with_yolo_version(YOLOVersion::V5) // YOLOVersion: V5, V6, V7, V8, V9, V10, RTDETR
.with_yolo_task(YOLOTask::Classify) // YOLOTask: Classify, Detect, Pose, Segment, Obb
.with_model("xxxx.onnx")?;
```
</details>
<details open>
<summary>Cutomized your own YOLO model</summary>
```Rust
// This config is for YOLOv8-Segment
use usls::{AnchorsPosition, BoxType, ClssType, YOLOPreds};
let options = Options::default()
.with_yolo_preds(
YOLOPreds {
bbox: Some(BoxType::Cxcywh),
clss: ClssType::Clss,
coefs: Some(true),
anchors: Some(AnchorsPosition::After),
..Default::default()
}
)
.with_model("xxxx.onnx")?;
```
</details>
## Other YOLOv8 Solution Models
| Model | Weights | Datasets|
|:---------------------: | :--------------------------: | :-------------------------------: |
| Face-Landmark Detection | [yolov8-face-dyn-f16](https://github.com/jamjamjon/assets/releases/download/v0.0.1/yolov8-face-dyn-f16.onnx) | |
| Head Detection | [yolov8-head-f16](https://github.com/jamjamjon/assets/releases/download/v0.0.1/yolov8-head-f16.onnx) | |
| Fall Detection | [yolov8-falldown-f16](https://github.com/jamjamjon/assets/releases/download/v0.0.1/yolov8-falldown-f16.onnx) | |
| Trash Detection | [yolov8-plastic-bag-f16](https://github.com/jamjamjon/assets/releases/download/v0.0.1/yolov8-plastic-bag-f16.onnx) | |
| FaceParsing | [face-parsing-dyn](https://github.com/jamjamjon/assets/releases/download/v0.0.1/face-parsing-dyn.onnx) | [CelebAMask-HQ](https://github.com/switchablenorms/CelebAMask-HQ/tree/master/face_parsing)<br />[[Processed YOLO labels]](https://github.com/jamjamjon/assets/releases/download/v0.0.1/CelebAMask-HQ-YOLO-Labels.zip)[[Python Script]](https://github.com/jamjamjon/assets/releases/download/v0.0.1/CelebAMask-HQ-YOLO-Labels.zip) |
## Export ONNX Models
<details close>
<summary>YOLOv5</summary>
[Here](https://docs.ultralytics.com/yolov5/tutorials/model_export/)
</details>
<details close>
<summary>YOLOv6</summary>
[Here](https://github.com/meituan/YOLOv6/tree/main/deploy/ONNX)
</details>
<details close>
<summary>YOLOv7</summary>
[Here](https://github.com/WongKinYiu/yolov7?tab=readme-ov-file#export)
</details>
<details close>
<summary>YOLOv8</summary>
```Shell
pip install -U ultralytics
# export onnx model with dynamic shapes
yolo export model=yolov8m.pt format=onnx simplify dynamic
yolo export model=yolov8m-cls.pt format=onnx simplify dynamic
yolo export model=yolov8m-pose.pt format=onnx simplify dynamic
yolo export model=yolov8m-seg.pt format=onnx simplify dynamic
yolo export model=yolov8m-obb.pt format=onnx simplify dynamic
# export onnx model with fixed shapes
yolo export model=yolov8m.pt format=onnx simplify
yolo export model=yolov8m-cls.pt format=onnx simplify
yolo export model=yolov8m-pose.pt format=onnx simplify
yolo export model=yolov8m-seg.pt format=onnx simplify
yolo export model=yolov8m-obb.pt format=onnx simplify
```
</details>
<details close>
<summary>YOLOv9</summary>
[Here](https://github.com/WongKinYiu/yolov9/blob/main/export.py)
</details>
<details close>
<summary>YOLOv10</summary>
[Here](https://github.com/THU-MIG/yolov10#export)
</details>

BIN
examples/yolo/demos/cls.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 453 KiB

BIN
examples/yolo/demos/det.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 451 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 105 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 286 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 321 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 291 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 546 KiB

BIN
examples/yolo/demos/obb.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 552 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 457 KiB

BIN
examples/yolo/demos/seg.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 391 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 367 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 453 KiB

180
examples/yolo/main.rs Normal file
View File

@ -0,0 +1,180 @@
use anyhow::Result;
use clap::Parser;
use usls::{coco, models::YOLO, Annotator, DataLoader, Options, Vision, YOLOTask, YOLOVersion};
#[derive(Parser, Clone)]
#[command(author, version, about, long_about = None)]
pub struct Args {
#[arg(long)]
pub model: Option<String>,
#[arg(long, default_value_t = String::from("./assets/bus.jpg"))]
pub source: String,
#[arg(long, value_enum, default_value_t = YOLOTask::Detect)]
pub task: YOLOTask,
#[arg(long, value_enum, default_value_t = YOLOVersion::V8)]
pub version: YOLOVersion,
#[arg(long, default_value_t = 224)]
pub width_min: isize,
#[arg(long, default_value_t = 640)]
pub width: isize,
#[arg(long, default_value_t = 800)]
pub width_max: isize,
#[arg(long, default_value_t = 224)]
pub height_min: isize,
#[arg(long, default_value_t = 640)]
pub height: isize,
#[arg(long, default_value_t = 800)]
pub height_max: isize,
#[arg(long, default_value_t = 80)]
pub nc: usize,
#[arg(long)]
pub trt: bool,
#[arg(long)]
pub cuda: bool,
#[arg(long)]
pub half: bool,
#[arg(long)]
pub coreml: bool,
#[arg(long, default_value_t = 0)]
pub device_id: usize,
#[arg(long)]
pub profile: bool,
#[arg(long)]
pub no_plot: bool,
}
fn main() -> Result<()> {
let args = Args::parse();
// build options
let options = Options::default();
// version & task
let options =
match args.version {
YOLOVersion::V5 => {
match args.task {
YOLOTask::Classify => options
.with_model(&args.model.unwrap_or("yolov5n-cls-dyn.onnx".to_string()))?,
YOLOTask::Detect => {
options.with_model(&args.model.unwrap_or("yolov5n-dyn.onnx".to_string()))?
}
YOLOTask::Segment => options
.with_model(&args.model.unwrap_or("yolov5n-seg-dyn.onnx".to_string()))?,
t => anyhow::bail!("Task: {t:?} is unsupported for {:?}", args.version),
}
}
YOLOVersion::V6 => match args.task {
YOLOTask::Detect => options
.with_model(&args.model.unwrap_or("yolov6n-dyn.onnx".to_string()))?
.with_nc(args.nc),
t => anyhow::bail!("Task: {t:?} is unsupported for {:?}", args.version),
},
YOLOVersion::V7 => match args.task {
YOLOTask::Detect => options
.with_model(&args.model.unwrap_or("yolov7-tiny-dyn.onnx".to_string()))?
.with_nc(args.nc),
t => anyhow::bail!("Task: {t:?} is unsupported for {:?}", args.version),
},
YOLOVersion::V8 => {
match args.task {
YOLOTask::Classify => options
.with_model(&args.model.unwrap_or("yolov8m-cls-dyn.onnx".to_string()))?,
YOLOTask::Detect => {
options.with_model(&args.model.unwrap_or("yolov8m-dyn.onnx".to_string()))?
}
YOLOTask::Segment => options
.with_model(&args.model.unwrap_or("yolov8m-seg-dyn.onnx".to_string()))?,
YOLOTask::Pose => options
.with_model(&args.model.unwrap_or("yolov8m-pose-dyn.onnx".to_string()))?,
YOLOTask::Obb => options
.with_model(&args.model.unwrap_or("yolov8m-obb-dyn.onnx".to_string()))?,
}
}
YOLOVersion::V9 => match args.task {
YOLOTask::Detect => options
.with_model(&args.model.unwrap_or("yolov9-c-dyn-f16.onnx".to_string()))?,
t => anyhow::bail!("Task: {t:?} is unsupported for {:?}", args.version),
},
YOLOVersion::V10 => match args.task {
YOLOTask::Detect => {
options.with_model(&args.model.unwrap_or("yolov10n-dyn.onnx".to_string()))?
}
t => anyhow::bail!("Task: {t:?} is unsupported for {:?}", args.version),
},
YOLOVersion::RTDETR => match args.task {
YOLOTask::Detect => {
options.with_model(&args.model.unwrap_or("rtdetr-l-f16.onnx".to_string()))?
}
t => anyhow::bail!("Task: {t:?} is unsupported for {:?}", args.version),
},
}
.with_yolo_version(args.version)
.with_yolo_task(args.task);
// device
let options = if args.cuda {
options.with_cuda(args.device_id)
} else if args.trt {
let options = options.with_trt(args.device_id);
if args.half {
options.with_fp16(true)
} else {
options
}
} else if args.coreml {
options.with_coreml(args.device_id)
} else {
options.with_cpu()
};
let options = options
.with_i00((1, 1, 4).into())
.with_i02((args.height_min, args.height, args.height_max).into())
.with_i03((args.width_min, args.width, args.width_max).into())
.with_confs(&[0.2, 0.15]) // class_0: 0.4, others: 0.15
// .with_names(&coco::NAMES_80)
.with_names2(&coco::KEYPOINTS_NAMES_17)
.with_profile(args.profile);
let mut model = YOLO::new(options)?;
// build dataloader
let dl = DataLoader::default()
.with_batch(model.batch() as _)
.load(args.source)?;
// build annotator
let annotator = Annotator::default()
.with_skeletons(&coco::SKELETONS_16)
.with_bboxes_thickness(7)
.without_masks(true) // No masks plotting.
.with_saveout("YOLO-Series");
// run & annotate
for (xs, _paths) in dl {
// let ys = model.run(&xs)?; // way one
let ys = model.forward(&xs, args.profile)?; // way two
if !args.no_plot {
annotator.annotate(&xs, &ys);
}
}
Ok(())
}