Initial media depth project backup
This commit is contained in:
Codex
134
README.md
Normal file
134
README.md
Normal file
@@ -0,0 +1,134 @@
|
||||
# Media Depth
|
||||
|
||||
这个项目用于批量生成图片/视频深度图，并把深度图转换成 PLY 点云。当前包含三套 Depth Anything 代码：
|
||||
|
||||
- `Depth-Anything-V1-main/`：Depth Anything V1，根目录批处理脚本会调用其中的 `run.py`。
|
||||
- `Depth-Anything-V2-main/`：Depth Anything V2，根目录批处理脚本会调用其中的 `run.py`。
|
||||
- `Depth-Anything-3-main-待处理/`：Depth Anything 3，支持图片、图片目录、视频、COLMAP、GLB/NPZ 等导出。
|
||||
- `Data/`：默认输入样例目录。
|
||||
- `1_Batch_Gen_All_Depth(跑之前删除结果文件夹).py`：批量跑 V1/V2 各 encoder，输出灰度和彩色深度图对比。
|
||||
- `2_Batch_Gen_All_PLY.py`：扫描批量深度图结果，把灰度深度图批量转为 PLY。
|
||||
- `Tool_Gen_3d_points_Cloud.py`：单组原图+深度图转 PLY 的底层工具。
|
||||
|
||||
## 环境
|
||||
|
||||
推荐使用独立 conda 环境：
|
||||
|
||||
```bash
|
||||
conda create -n Media_Depth python=3.12 -y
|
||||
conda activate Media_Depth
|
||||
|
||||
pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu118
|
||||
pip install numpy==1.26.4 opencv-python==4.11.0.86 matplotlib tqdm huggingface_hub imageio einops requests typer omegaconf safetensors moviepy==1.0.3 plyfile trimesh fastapi uvicorn evo e3nn pycolmap open3d pre-commit
|
||||
pip install gradio==4.29.0 gradio_imageslider pillow_heif==0.22.0
|
||||
pip install xformers==0.0.31.post1 --no-deps
|
||||
|
||||
cd Depth-Anything-3-main-待处理
|
||||
pip install -e . --no-deps
|
||||
|
||||
conda install -n Media_Depth -c nvidia cuda-nvcc=11.8 -y
|
||||
conda install -n Media_Depth -c nvidia/label/cuda-11.8.0 cuda-toolkit -y
|
||||
export CUDA_HOME="$CONDA_PREFIX"
|
||||
export PATH="$CONDA_PREFIX/bin:$PATH"
|
||||
pip install ninja jaxtyping
|
||||
pip install --no-build-isolation "git+https://github.com/nerfstudio-project/gsplat.git@0b4dddf04cb687367602c01196913cde6a743d70"
|
||||
```
|
||||
|
||||
如果只是跑 V1/V2、PLY 或 DA3 的普通深度/NPZ/GLB 输出，`gsplat` 不是必需的；如果需要 DA3 的 Gaussian/GS 相关导出，再安装它。`gsplat` 编译时必须让 `nvcc -V` 显示 CUDA 11.8，否则会和 `torch==2.7.1+cu118` 不匹配。
|
||||
|
||||
GPU 检查：
|
||||
|
||||
```bash
|
||||
nvidia-smi
|
||||
python -c "import torch; print(torch.__version__); print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0))"
|
||||
```
|
||||
|
||||
## 权重下载
|
||||
|
||||
权重文件不纳入 Git 备份。下载后按下面路径放置：
|
||||
|
||||
### V1
|
||||
|
||||
放到 `Depth-Anything-V1-main/checkpoints/`：
|
||||
|
||||
- `depth_anything_vits14.pth`：https://huggingface.co/spaces/LiheYoung/Depth-Anything/blob/main/checkpoints/depth_anything_vits14.pth
|
||||
- `depth_anything_vitb14.pth`：https://huggingface.co/spaces/LiheYoung/Depth-Anything/blob/main/checkpoints/depth_anything_vitb14.pth
|
||||
- `depth_anything_vitl14.pth`：https://huggingface.co/spaces/LiheYoung/Depth-Anything/blob/main/checkpoints/depth_anything_vitl14.pth
|
||||
|
||||
### V2
|
||||
|
||||
放到 `Depth-Anything-V2-main/checkpoints/`：
|
||||
|
||||
- `depth_anything_v2_vits.pth`：https://huggingface.co/depth-anything/Depth-Anything-V2-Small
|
||||
- `depth_anything_v2_vitb.pth`：https://huggingface.co/depth-anything/Depth-Anything-V2-Base
|
||||
- `depth_anything_v2_vitl.pth`：https://huggingface.co/depth-anything/Depth-Anything-V2-Large
|
||||
- `depth_anything_v2_vitg.pth`：https://huggingface.co/depth-anything/Depth-Anything-V2-Giant
|
||||
|
||||
### DA3
|
||||
|
||||
每个模型目录放到 `Depth-Anything-3-main-待处理/checkpoints/`，目录中应包含 `config.json` 和 `model.safetensors`：
|
||||
|
||||
- `DA3-SMALL`：https://huggingface.co/depth-anything/DA3-SMALL
|
||||
- `DA3-BASE`：https://huggingface.co/depth-anything/DA3-BASE
|
||||
- `DA3-LARGE`：https://huggingface.co/depth-anything/DA3-LARGE
|
||||
- `DA3-GIANT`：https://huggingface.co/depth-anything/DA3-GIANT
|
||||
- `DA3MONO-LARGE`：https://huggingface.co/depth-anything/DA3MONO-LARGE
|
||||
- `DA3METRIC-LARGE`：https://huggingface.co/depth-anything/DA3METRIC-LARGE
|
||||
- `DA3NESTED-GIANT-LARGE`：https://huggingface.co/depth-anything/DA3NESTED-GIANT-LARGE
|
||||
|
||||
## 常用命令
|
||||
|
||||
批量生成 V1/V2 深度图：
|
||||
|
||||
```bash
|
||||
python "1_Batch_Gen_All_Depth(跑之前删除结果文件夹).py" --img-path ./Data
|
||||
```
|
||||
|
||||
批量生成 PLY：
|
||||
|
||||
```bash
|
||||
python 2_Batch_Gen_All_PLY.py --img-path ./Data --z-scale 0.7 --brightness 1.0 --saturation 1.0 --gamma 1.0
|
||||
```
|
||||
|
||||
DA3 单张图片：
|
||||
|
||||
```bash
|
||||
cd Depth-Anything-3-main-待处理
|
||||
da3 image ../Data/09-19-54.mp4.00_02_12_09.Still001.png \
|
||||
--model-dir checkpoints/DA3-SMALL \
|
||||
--device cuda \
|
||||
--export-format mini_npz \
|
||||
--export-dir workspace/gallery/test_image \
|
||||
--auto-cleanup
|
||||
```
|
||||
|
||||
DA3 自动识别图片/目录/视频：
|
||||
|
||||
```bash
|
||||
cd Depth-Anything-3-main-待处理
|
||||
da3 auto ../Data/09-19-54.mp4.00_02_12_09.Still001.png \
|
||||
--model-dir checkpoints/DA3-SMALL \
|
||||
--device cuda \
|
||||
--export-format mini_npz-glb \
|
||||
--export-dir workspace/gallery/test_auto \
|
||||
--auto-cleanup
|
||||
```
|
||||
|
||||
DA3 视频：
|
||||
|
||||
```bash
|
||||
cd Depth-Anything-3-main-待处理
|
||||
da3 video path/to/video.mp4 \
|
||||
--fps 1 \
|
||||
--model-dir checkpoints/DA3-SMALL \
|
||||
--device cuda \
|
||||
--export-format mini_npz \
|
||||
--export-dir workspace/gallery/test_video \
|
||||
--auto-cleanup
|
||||
```
|
||||
|
||||
如果遇到 Matplotlib 缓存目录不可写，可以临时指定：
|
||||
|
||||
```bash
|
||||
export MPLCONFIGDIR=/tmp/matplotlib-media-depth
|
||||
```
|
||||
Reference in New Issue
Block a user