135 lines
4.9 KiB
Markdown
135 lines
4.9 KiB
Markdown
# Media Depth
|
||
|
||
这个项目用于批量生成图片/视频深度图，并把深度图转换成 PLY 点云。当前包含三套 Depth Anything 代码：
|
||
|
||
- `Depth-Anything-V1-main/`：Depth Anything V1，根目录批处理脚本会调用其中的 `run.py`。
|
||
- `Depth-Anything-V2-main/`：Depth Anything V2，根目录批处理脚本会调用其中的 `run.py`。
|
||
- `Depth-Anything-V3-main/`：Depth Anything 3，支持图片、图片目录、视频、COLMAP、GLB/NPZ 等导出。
|
||
- `Data/`：默认输入样例目录。
|
||
- `1_Batch_Gen_All_Depth(跑之前删除结果文件夹).py`：批量跑 V1/V2 各 encoder，输出灰度和彩色深度图对比。
|
||
- `2_Batch_Gen_All_PLY.py`：扫描批量深度图结果，把灰度深度图批量转为 PLY。
|
||
- `Tool_Gen_3d_points_Cloud.py`：单组原图+深度图转 PLY 的底层工具。
|
||
|
||
## 环境
|
||
|
||
推荐使用独立 conda 环境：
|
||
|
||
```bash
|
||
conda create -n Media_Depth python=3.12 -y
|
||
conda activate Media_Depth
|
||
|
||
pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu118
|
||
pip install numpy==1.26.4 opencv-python==4.11.0.86 matplotlib tqdm huggingface_hub imageio einops requests typer omegaconf safetensors moviepy==1.0.3 plyfile trimesh fastapi uvicorn evo e3nn pycolmap open3d pre-commit
|
||
pip install gradio==4.29.0 gradio_imageslider pillow_heif==0.22.0
|
||
pip install xformers==0.0.31.post1 --no-deps
|
||
|
||
cd Depth-Anything-V3-main
|
||
pip install -e . --no-deps
|
||
|
||
conda install -n Media_Depth -c nvidia cuda-nvcc=11.8 -y
|
||
conda install -n Media_Depth -c nvidia/label/cuda-11.8.0 cuda-toolkit -y
|
||
export CUDA_HOME="$CONDA_PREFIX"
|
||
export PATH="$CONDA_PREFIX/bin:$PATH"
|
||
pip install ninja jaxtyping
|
||
pip install --no-build-isolation "git+https://github.com/nerfstudio-project/gsplat.git@0b4dddf04cb687367602c01196913cde6a743d70"
|
||
```
|
||
|
||
如果只是跑 V1/V2、PLY 或 DA3 的普通深度/NPZ/GLB 输出，`gsplat` 不是必需的；如果需要 DA3 的 Gaussian/GS 相关导出，再安装它。`gsplat` 编译时必须让 `nvcc -V` 显示 CUDA 11.8，否则会和 `torch==2.7.1+cu118` 不匹配。
|
||
|
||
GPU 检查：
|
||
|
||
```bash
|
||
nvidia-smi
|
||
python -c "import torch; print(torch.__version__); print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0))"
|
||
```
|
||
|
||
## 权重下载
|
||
|
||
权重文件不纳入 Git 备份。下载后按下面路径放置：
|
||
|
||
### V1
|
||
|
||
放到 `Depth-Anything-V1-main/checkpoints/`：
|
||
|
||
- `depth_anything_vits14.pth`：https://huggingface.co/spaces/LiheYoung/Depth-Anything/blob/main/checkpoints/depth_anything_vits14.pth
|
||
- `depth_anything_vitb14.pth`：https://huggingface.co/spaces/LiheYoung/Depth-Anything/blob/main/checkpoints/depth_anything_vitb14.pth
|
||
- `depth_anything_vitl14.pth`：https://huggingface.co/spaces/LiheYoung/Depth-Anything/blob/main/checkpoints/depth_anything_vitl14.pth
|
||
|
||
### V2
|
||
|
||
放到 `Depth-Anything-V2-main/checkpoints/`：
|
||
|
||
- `depth_anything_v2_vits.pth`：https://huggingface.co/depth-anything/Depth-Anything-V2-Small
|
||
- `depth_anything_v2_vitb.pth`：https://huggingface.co/depth-anything/Depth-Anything-V2-Base
|
||
- `depth_anything_v2_vitl.pth`：https://huggingface.co/depth-anything/Depth-Anything-V2-Large
|
||
- `depth_anything_v2_vitg.pth`：https://huggingface.co/depth-anything/Depth-Anything-V2-Giant
|
||
|
||
### DA3
|
||
|
||
每个模型目录放到 `Depth-Anything-V3-main/checkpoints/`，目录中应包含 `config.json` 和 `model.safetensors`：
|
||
|
||
- `DA3-SMALL`：https://huggingface.co/depth-anything/DA3-SMALL
|
||
- `DA3-BASE`：https://huggingface.co/depth-anything/DA3-BASE
|
||
- `DA3-LARGE`：https://huggingface.co/depth-anything/DA3-LARGE
|
||
- `DA3-GIANT`：https://huggingface.co/depth-anything/DA3-GIANT
|
||
- `DA3MONO-LARGE`：https://huggingface.co/depth-anything/DA3MONO-LARGE
|
||
- `DA3METRIC-LARGE`：https://huggingface.co/depth-anything/DA3METRIC-LARGE
|
||
- `DA3NESTED-GIANT-LARGE`：https://huggingface.co/depth-anything/DA3NESTED-GIANT-LARGE
|
||
|
||
## 常用命令
|
||
|
||
批量生成 V1/V2 深度图：
|
||
|
||
```bash
|
||
python "1_Batch_Gen_All_Depth(跑之前删除结果文件夹).py" --img-path ./Data
|
||
```
|
||
|
||
批量生成 PLY：
|
||
|
||
```bash
|
||
python 2_Batch_Gen_All_PLY.py --img-path ./Data --z-scale 0.7 --brightness 1.0 --saturation 1.0 --gamma 1.0
|
||
```
|
||
|
||
DA3 单张图片：
|
||
|
||
```bash
|
||
cd Depth-Anything-V3-main
|
||
da3 image ../Data/09-19-54.mp4.00_02_12_09.Still001.png \
|
||
--model-dir checkpoints/DA3-SMALL \
|
||
--device cuda \
|
||
--export-format mini_npz \
|
||
--export-dir workspace/gallery/test_image \
|
||
--auto-cleanup
|
||
```
|
||
|
||
DA3 自动识别图片/目录/视频：
|
||
|
||
```bash
|
||
cd Depth-Anything-V3-main
|
||
da3 auto ../Data/09-19-54.mp4.00_02_12_09.Still001.png \
|
||
--model-dir checkpoints/DA3-SMALL \
|
||
--device cuda \
|
||
--export-format mini_npz-glb \
|
||
--export-dir workspace/gallery/test_auto \
|
||
--auto-cleanup
|
||
```
|
||
|
||
DA3 视频：
|
||
|
||
```bash
|
||
cd Depth-Anything-V3-main
|
||
da3 video path/to/video.mp4 \
|
||
--fps 1 \
|
||
--model-dir checkpoints/DA3-SMALL \
|
||
--device cuda \
|
||
--export-format mini_npz \
|
||
--export-dir workspace/gallery/test_video \
|
||
--auto-cleanup
|
||
```
|
||
|
||
如果遇到 Matplotlib 缓存目录不可写，可以临时指定：
|
||
|
||
```bash
|
||
export MPLCONFIGDIR=/tmp/matplotlib-media-depth
|
||
```
|