first commit
This commit is contained in:
admin
76
Seg_All_In_One_MMSeg/configs/twins/README.md
Normal file
76
Seg_All_In_One_MMSeg/configs/twins/README.md
Normal file
@@ -0,0 +1,76 @@
|
||||
# Twins
|
||||
|
||||
> [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://arxiv.org/pdf/2104.13840.pdf)
|
||||
|
||||
## Introduction
|
||||
|
||||
<!-- [BACKBONE] -->
|
||||
|
||||
<a href = "https://github.com/Meituan-AutoML/Twins">Official Repo</a>
|
||||
|
||||
<a href="https://github.com/open-mmlab/mmsegmentation/blob/v0.20.0/mmseg/models/backbones/twins.py#L352">Code Snippet</a>
|
||||
|
||||
## Abstract
|
||||
|
||||
<!-- [ABSTRACT] -->
|
||||
|
||||
Very recently, a variety of vision transformer architectures for dense prediction tasks have been proposed and they show that the design of spatial attention is critical to their success in these tasks. In this work, we revisit the design of the spatial attention and demonstrate that a carefully-devised yet simple spatial attention mechanism performs favourably against the state-of-the-art schemes. As a result, we propose two vision transformer architectures, namely, Twins-PCPVT and Twins-SVT. Our proposed architectures are highly-efficient and easy to implement, only involving matrix multiplications that are highly optimized in modern deep learning frameworks. More importantly, the proposed architectures achieve excellent performance on a wide range of visual tasks, including image level classification as well as dense detection and segmentation. The simplicity and strong performance suggest that our proposed architectures may serve as stronger backbones for many vision tasks. Our code is released at [this https URL](https://github.com/Meituan-AutoML/Twins).
|
||||
|
||||
<!-- [IMAGE] -->
|
||||
|
||||
<div align=center>
|
||||
<img src="https://user-images.githubusercontent.com/24582831/145021310-57826cf5-5e03-4c7c-9081-ffa744bdae27.png" width="80%"/>
|
||||
</div>
|
||||
|
||||
## Usage
|
||||
|
||||
We have provided pretrained models converted from [official repo](https://github.com/Meituan-AutoML/Twins).
|
||||
|
||||
If you want to convert keys on your own to use official repositories' pre-trained models, we also provide a script [`twins2mmseg.py`](../../tools/model_converters/twins2mmseg.py) in the tools directory to convert the key of models from [the official repo](https://github.com/Meituan-AutoML/Twins) to MMSegmentation style.
|
||||
|
||||
```shell
|
||||
python tools/model_converters/twins2mmseg.py ${PRETRAIN_PATH} ${STORE_PATH} ${MODEL_TYPE}
|
||||
```
|
||||
|
||||
This script convert `pcpvt` or `svt` pretrained model from `PRETRAIN_PATH` and store the converted model in `STORE_PATH`.
|
||||
|
||||
For example,
|
||||
|
||||
```shell
|
||||
python tools/model_converters/twins2mmseg.py ./alt_gvt_base.pth ./pretrained/alt_gvt_base.pth svt
|
||||
```
|
||||
|
||||
## Results and models
|
||||
|
||||
### ADE20K
|
||||
|
||||
| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
|
||||
| ------- | ------------------- | --------- | ------- | -------- | -------------- | ------ | ----- | ------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| FPN | Twins-PCPVT-S | 512x512 | 80000 | 6.60 | 27.15 | V100 | 43.26 | 44.11 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/twins/twins_pcpvt-s_fpn_fpnhead_8xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-s_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_pcpvt-s_fpn_fpnhead_8x4_512x512_80k_ade20k_20211201_204132-41acd132.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-s_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_pcpvt-s_fpn_fpnhead_8x4_512x512_80k_ade20k_20211201_204132.log.json) |
|
||||
| UPerNet | Twins-PCPVT-S | 512x512 | 160000 | 9.67 | 14.24 | V100 | 46.04 | 46.92 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/twins/twins_pcpvt-s_uperhead_8xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-s_uperhead_8x4_512x512_160k_ade20k/twins_pcpvt-s_uperhead_8x4_512x512_160k_ade20k_20211201_233537-8e99c07a.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-s_uperhead_8x4_512x512_160k_ade20k/twins_pcpvt-s_uperhead_8x4_512x512_160k_ade20k_20211201_233537.log.json) |
|
||||
| FPN | Twins-PCPVT-B | 512x512 | 80000 | 8.41 | 19.67 | V100 | 45.66 | 46.48 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/twins/twins_pcpvt-b_fpn_fpnhead_8xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-b_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_pcpvt-b_fpn_fpnhead_8x4_512x512_80k_ade20k_20211130_141019-d396db72.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-b_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_pcpvt-b_fpn_fpnhead_8x4_512x512_80k_ade20k_20211130_141019.log.json) |
|
||||
| UPerNet | Twins-PCPVT-B (8x2) | 512x512 | 160000 | 6.46 | 12.04 | V100 | 47.91 | 48.64 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/twins/twins_pcpvt-b_uperhead_8xb2-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-b_uperhead_8x2_512x512_160k_ade20k/twins_pcpvt-b_uperhead_8x2_512x512_160k_ade20k_20211130_141020-02094ea5.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-b_uperhead_8x2_512x512_160k_ade20k/twins_pcpvt-b_uperhead_8x2_512x512_160k_ade20k_20211130_141020.log.json) |
|
||||
| FPN | Twins-PCPVT-L | 512x512 | 80000 | 10.78 | 14.32 | V100 | 45.94 | 46.70 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/twins/twins_pcpvt-l_fpn_fpnhead_8xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-l_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_pcpvt-l_fpn_fpnhead_8x4_512x512_80k_ade20k_20211201_105226-bc6d61dc.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-l_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_pcpvt-l_fpn_fpnhead_8x4_512x512_80k_ade20k_20211201_105226.log.json) |
|
||||
| UPerNet | Twins-PCPVT-L (8x2) | 512x512 | 160000 | 7.82 | 10.70 | V100 | 49.35 | 50.08 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/twins/twins_pcpvt-l_uperhead_8xb2-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-l_uperhead_8x2_512x512_160k_ade20k/twins_pcpvt-l_uperhead_8x2_512x512_160k_ade20k_20211201_075053-c6095c07.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-l_uperhead_8x2_512x512_160k_ade20k/twins_pcpvt-l_uperhead_8x2_512x512_160k_ade20k_20211201_075053.log.json) |
|
||||
| FPN | Twins-SVT-S | 512x512 | 80000 | 5.80 | 29.79 | V100 | 44.47 | 45.42 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/twins/twins_svt-s_fpn_fpnhead_8xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-s_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_svt-s_fpn_fpnhead_8x4_512x512_80k_ade20k_20211130_141006-0a0d3317.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-s_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_svt-s_fpn_fpnhead_8x4_512x512_80k_ade20k_20211130_141006.log.json) |
|
||||
| UPerNet | SVT-S (8x2) | 512x512 | 160000 | 4.93 | 15.09 | V100 | 46.08 | 46.96 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/twins/twins_svt-s_uperhead_8xb2-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-s_uperhead_8x2_512x512_160k_ade20k/twins_svt-s_uperhead_8x2_512x512_160k_ade20k_20211130_141005-e48a2d94.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-s_uperhead_8x2_512x512_160k_ade20k/twins_svt-s_uperhead_8x2_512x512_160k_ade20k_20211130_141005.log.json) |
|
||||
| FPN | Twins-SVT-B | 512x512 | 80000 | 8.75 | 21.10 | V100 | 46.77 | 47.47 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/twins/twins_svt-b_fpn_fpnhead_8xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-b_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_svt-b_fpn_fpnhead_8x4_512x512_80k_ade20k_20211201_113849-88b2907c.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-b_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_svt-b_fpn_fpnhead_8x4_512x512_80k_ade20k_20211201_113849.log.json) |
|
||||
| UPerNet | Twins-SVT-B (8x2) | 512x512 | 160000 | 6.77 | 12.66 | V100 | 48.04 | 48.87 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/twins/twins_svt-b_uperhead_8xb2-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-b_uperhead_8x2_512x512_160k_ade20k/twins_svt-b_uperhead_8x2_512x512_160k_ade20k_20211202_040826-0943a1f1.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-b_uperhead_8x2_512x512_160k_ade20k/twins_svt-b_uperhead_8x2_512x512_160k_ade20k_20211202_040826.log.json) |
|
||||
| FPN | Twins-SVT-L | 512x512 | 80000 | 11.20 | 17.80 | V100 | 46.55 | 47.74 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/twins/twins_svt-l_fpn_fpnhead_8xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-l_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_svt-l_fpn_fpnhead_8x4_512x512_80k_ade20k_20211130_141005-1d59bee2.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-l_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_svt-l_fpn_fpnhead_8x4_512x512_80k_ade20k_20211130_141005.log.json) |
|
||||
| UPerNet | Twins-SVT-L (8x2) | 512x512 | 160000 | 8.41 | 10.73 | V100 | 49.65 | 50.63 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/twins/twins_pcpvt-l_uperhead_8xb2-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-l_uperhead_8x2_512x512_160k_ade20k/twins_svt-l_uperhead_8x2_512x512_160k_ade20k_20211130_141005-3e2cae61.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-l_uperhead_8x2_512x512_160k_ade20k/twins_svt-l_uperhead_8x2_512x512_160k_ade20k_20211130_141005.log.json) |
|
||||
|
||||
Note:
|
||||
|
||||
- `8x2` means 8 GPUs with 2 samples per GPU in training. Default setting of Twins on ADE20K is 8 GPUs with 4 samples per GPU in training.
|
||||
- `UPerNet` and `FPN` are decoder heads utilized in corresponding Twins model, which is `UPerHead` and `FPNHead`, respectively. Specifically, models in [official repo](https://github.com/Meituan-AutoML/Twins) all use `UPerHead`.
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@article{chu2021twins,
|
||||
title={Twins: Revisiting spatial attention design in vision transformers},
|
||||
author={Chu, Xiangxiang and Tian, Zhi and Wang, Yuqing and Zhang, Bo and Ren, Haibing and Wei, Xiaolin and Xia, Huaxia and Shen, Chunhua},
|
||||
journal={arXiv preprint arXiv:2104.13840},
|
||||
year={2021}altgvt
|
||||
}
|
||||
```
|
||||
289
Seg_All_In_One_MMSeg/configs/twins/metafile.yaml
Normal file
289
Seg_All_In_One_MMSeg/configs/twins/metafile.yaml
Normal file
@@ -0,0 +1,289 @@
|
||||
Models:
|
||||
- Name: twins_pcpvt-s_fpn_fpnhead_8xb4-80k_ade20k-512x512
|
||||
In Collection: FPN
|
||||
Results:
|
||||
Task: Semantic Segmentation
|
||||
Dataset: ADE20K
|
||||
Metrics:
|
||||
mIoU: 43.26
|
||||
mIoU(ms+flip): 44.11
|
||||
Config: configs/twins/twins_pcpvt-s_fpn_fpnhead_8xb4-80k_ade20k-512x512.py
|
||||
Metadata:
|
||||
Training Data: ADE20K
|
||||
Batch Size: 32
|
||||
Architecture:
|
||||
- Twins-PCPVT-S
|
||||
- FPN
|
||||
Training Resources: 8x V100 GPUS
|
||||
Memory (GB): 6.6
|
||||
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-s_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_pcpvt-s_fpn_fpnhead_8x4_512x512_80k_ade20k_20211201_204132-41acd132.pth
|
||||
Training log: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-s_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_pcpvt-s_fpn_fpnhead_8x4_512x512_80k_ade20k_20211201_204132.log.json
|
||||
Paper:
|
||||
Title: 'Twins: Revisiting the Design of Spatial Attention in Vision Transformers'
|
||||
URL: https://arxiv.org/pdf/2104.13840.pdf
|
||||
Code: https://github.com/open-mmlab/mmsegmentation/blob/v0.20.0/mmseg/models/backbones/twins.py#L352
|
||||
Framework: PyTorch
|
||||
- Name: twins_pcpvt-s_uperhead_8xb4-160k_ade20k-512x512
|
||||
In Collection: UPerNet
|
||||
Results:
|
||||
Task: Semantic Segmentation
|
||||
Dataset: ADE20K
|
||||
Metrics:
|
||||
mIoU: 46.04
|
||||
mIoU(ms+flip): 46.92
|
||||
Config: configs/twins/twins_pcpvt-s_uperhead_8xb4-160k_ade20k-512x512.py
|
||||
Metadata:
|
||||
Training Data: ADE20K
|
||||
Batch Size: 32
|
||||
Architecture:
|
||||
- Twins-PCPVT-S
|
||||
- UPerNet
|
||||
Training Resources: 8x V100 GPUS
|
||||
Memory (GB): 9.67
|
||||
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-s_uperhead_8x4_512x512_160k_ade20k/twins_pcpvt-s_uperhead_8x4_512x512_160k_ade20k_20211201_233537-8e99c07a.pth
|
||||
Training log: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-s_uperhead_8x4_512x512_160k_ade20k/twins_pcpvt-s_uperhead_8x4_512x512_160k_ade20k_20211201_233537.log.json
|
||||
Paper:
|
||||
Title: 'Twins: Revisiting the Design of Spatial Attention in Vision Transformers'
|
||||
URL: https://arxiv.org/pdf/2104.13840.pdf
|
||||
Code: https://github.com/open-mmlab/mmsegmentation/blob/v0.20.0/mmseg/models/backbones/twins.py#L352
|
||||
Framework: PyTorch
|
||||
- Name: twins_pcpvt-b_fpn_fpnhead_8xb4-80k_ade20k-512x512
|
||||
In Collection: FPN
|
||||
Results:
|
||||
Task: Semantic Segmentation
|
||||
Dataset: ADE20K
|
||||
Metrics:
|
||||
mIoU: 45.66
|
||||
mIoU(ms+flip): 46.48
|
||||
Config: configs/twins/twins_pcpvt-b_fpn_fpnhead_8xb4-80k_ade20k-512x512.py
|
||||
Metadata:
|
||||
Training Data: ADE20K
|
||||
Batch Size: 32
|
||||
Architecture:
|
||||
- Twins-PCPVT-B
|
||||
- FPN
|
||||
Training Resources: 8x V100 GPUS
|
||||
Memory (GB): 8.41
|
||||
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-b_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_pcpvt-b_fpn_fpnhead_8x4_512x512_80k_ade20k_20211130_141019-d396db72.pth
|
||||
Training log: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-b_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_pcpvt-b_fpn_fpnhead_8x4_512x512_80k_ade20k_20211130_141019.log.json
|
||||
Paper:
|
||||
Title: 'Twins: Revisiting the Design of Spatial Attention in Vision Transformers'
|
||||
URL: https://arxiv.org/pdf/2104.13840.pdf
|
||||
Code: https://github.com/open-mmlab/mmsegmentation/blob/v0.20.0/mmseg/models/backbones/twins.py#L352
|
||||
Framework: PyTorch
|
||||
- Name: twins_pcpvt-b_uperhead_8xb2-160k_ade20k-512x512
|
||||
In Collection: UPerNet
|
||||
Results:
|
||||
Task: Semantic Segmentation
|
||||
Dataset: ADE20K
|
||||
Metrics:
|
||||
mIoU: 47.91
|
||||
mIoU(ms+flip): 48.64
|
||||
Config: configs/twins/twins_pcpvt-b_uperhead_8xb2-160k_ade20k-512x512.py
|
||||
Metadata:
|
||||
Training Data: ADE20K
|
||||
Batch Size: 16
|
||||
Architecture:
|
||||
- Twins-PCPVT-B
|
||||
- UPerNet
|
||||
Training Resources: 8x V100 GPUS
|
||||
Memory (GB): 6.46
|
||||
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-b_uperhead_8x2_512x512_160k_ade20k/twins_pcpvt-b_uperhead_8x2_512x512_160k_ade20k_20211130_141020-02094ea5.pth
|
||||
Training log: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-b_uperhead_8x2_512x512_160k_ade20k/twins_pcpvt-b_uperhead_8x2_512x512_160k_ade20k_20211130_141020.log.json
|
||||
Paper:
|
||||
Title: 'Twins: Revisiting the Design of Spatial Attention in Vision Transformers'
|
||||
URL: https://arxiv.org/pdf/2104.13840.pdf
|
||||
Code: https://github.com/open-mmlab/mmsegmentation/blob/v0.20.0/mmseg/models/backbones/twins.py#L352
|
||||
Framework: PyTorch
|
||||
- Name: twins_pcpvt-l_fpn_fpnhead_8xb4-80k_ade20k-512x512
|
||||
In Collection: FPN
|
||||
Results:
|
||||
Task: Semantic Segmentation
|
||||
Dataset: ADE20K
|
||||
Metrics:
|
||||
mIoU: 45.94
|
||||
mIoU(ms+flip): 46.7
|
||||
Config: configs/twins/twins_pcpvt-l_fpn_fpnhead_8xb4-80k_ade20k-512x512.py
|
||||
Metadata:
|
||||
Training Data: ADE20K
|
||||
Batch Size: 32
|
||||
Architecture:
|
||||
- Twins-PCPVT-L
|
||||
- FPN
|
||||
Training Resources: 8x V100 GPUS
|
||||
Memory (GB): 10.78
|
||||
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-l_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_pcpvt-l_fpn_fpnhead_8x4_512x512_80k_ade20k_20211201_105226-bc6d61dc.pth
|
||||
Training log: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-l_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_pcpvt-l_fpn_fpnhead_8x4_512x512_80k_ade20k_20211201_105226.log.json
|
||||
Paper:
|
||||
Title: 'Twins: Revisiting the Design of Spatial Attention in Vision Transformers'
|
||||
URL: https://arxiv.org/pdf/2104.13840.pdf
|
||||
Code: https://github.com/open-mmlab/mmsegmentation/blob/v0.20.0/mmseg/models/backbones/twins.py#L352
|
||||
Framework: PyTorch
|
||||
- Name: twins_pcpvt-l_uperhead_8xb2-160k_ade20k-512x512
|
||||
In Collection: UPerNet
|
||||
Results:
|
||||
Task: Semantic Segmentation
|
||||
Dataset: ADE20K
|
||||
Metrics:
|
||||
mIoU: 49.35
|
||||
mIoU(ms+flip): 50.08
|
||||
Config: configs/twins/twins_pcpvt-l_uperhead_8xb2-160k_ade20k-512x512.py
|
||||
Metadata:
|
||||
Training Data: ADE20K
|
||||
Batch Size: 16
|
||||
Architecture:
|
||||
- Twins-PCPVT-L
|
||||
- UPerNet
|
||||
Training Resources: 8x V100 GPUS
|
||||
Memory (GB): 7.82
|
||||
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-l_uperhead_8x2_512x512_160k_ade20k/twins_pcpvt-l_uperhead_8x2_512x512_160k_ade20k_20211201_075053-c6095c07.pth
|
||||
Training log: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_pcpvt-l_uperhead_8x2_512x512_160k_ade20k/twins_pcpvt-l_uperhead_8x2_512x512_160k_ade20k_20211201_075053.log.json
|
||||
Paper:
|
||||
Title: 'Twins: Revisiting the Design of Spatial Attention in Vision Transformers'
|
||||
URL: https://arxiv.org/pdf/2104.13840.pdf
|
||||
Code: https://github.com/open-mmlab/mmsegmentation/blob/v0.20.0/mmseg/models/backbones/twins.py#L352
|
||||
Framework: PyTorch
|
||||
- Name: twins_svt-s_fpn_fpnhead_8xb4-80k_ade20k-512x512
|
||||
In Collection: FPN
|
||||
Results:
|
||||
Task: Semantic Segmentation
|
||||
Dataset: ADE20K
|
||||
Metrics:
|
||||
mIoU: 44.47
|
||||
mIoU(ms+flip): 45.42
|
||||
Config: configs/twins/twins_svt-s_fpn_fpnhead_8xb4-80k_ade20k-512x512.py
|
||||
Metadata:
|
||||
Training Data: ADE20K
|
||||
Batch Size: 32
|
||||
Architecture:
|
||||
- Twins-SVT-S
|
||||
- FPN
|
||||
Training Resources: 8x V100 GPUS
|
||||
Memory (GB): 5.8
|
||||
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-s_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_svt-s_fpn_fpnhead_8x4_512x512_80k_ade20k_20211130_141006-0a0d3317.pth
|
||||
Training log: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-s_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_svt-s_fpn_fpnhead_8x4_512x512_80k_ade20k_20211130_141006.log.json
|
||||
Paper:
|
||||
Title: 'Twins: Revisiting the Design of Spatial Attention in Vision Transformers'
|
||||
URL: https://arxiv.org/pdf/2104.13840.pdf
|
||||
Code: https://github.com/open-mmlab/mmsegmentation/blob/v0.20.0/mmseg/models/backbones/twins.py#L352
|
||||
Framework: PyTorch
|
||||
- Name: twins_svt-s_uperhead_8xb2-160k_ade20k-512x512
|
||||
In Collection: UPerNet
|
||||
Results:
|
||||
Task: Semantic Segmentation
|
||||
Dataset: ADE20K
|
||||
Metrics:
|
||||
mIoU: 46.08
|
||||
mIoU(ms+flip): 46.96
|
||||
Config: configs/twins/twins_svt-s_uperhead_8xb2-160k_ade20k-512x512.py
|
||||
Metadata:
|
||||
Training Data: ADE20K
|
||||
Batch Size: 16
|
||||
Architecture:
|
||||
- SVT-S
|
||||
- UPerNet
|
||||
Training Resources: 8x V100 GPUS
|
||||
Memory (GB): 4.93
|
||||
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-s_uperhead_8x2_512x512_160k_ade20k/twins_svt-s_uperhead_8x2_512x512_160k_ade20k_20211130_141005-e48a2d94.pth
|
||||
Training log: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-s_uperhead_8x2_512x512_160k_ade20k/twins_svt-s_uperhead_8x2_512x512_160k_ade20k_20211130_141005.log.json
|
||||
Paper:
|
||||
Title: 'Twins: Revisiting the Design of Spatial Attention in Vision Transformers'
|
||||
URL: https://arxiv.org/pdf/2104.13840.pdf
|
||||
Code: https://github.com/open-mmlab/mmsegmentation/blob/v0.20.0/mmseg/models/backbones/twins.py#L352
|
||||
Framework: PyTorch
|
||||
- Name: twins_svt-b_fpn_fpnhead_8xb4-80k_ade20k-512x512
|
||||
In Collection: FPN
|
||||
Results:
|
||||
Task: Semantic Segmentation
|
||||
Dataset: ADE20K
|
||||
Metrics:
|
||||
mIoU: 46.77
|
||||
mIoU(ms+flip): 47.47
|
||||
Config: configs/twins/twins_svt-b_fpn_fpnhead_8xb4-80k_ade20k-512x512.py
|
||||
Metadata:
|
||||
Training Data: ADE20K
|
||||
Batch Size: 32
|
||||
Architecture:
|
||||
- Twins-SVT-B
|
||||
- FPN
|
||||
Training Resources: 8x V100 GPUS
|
||||
Memory (GB): 8.75
|
||||
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-b_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_svt-b_fpn_fpnhead_8x4_512x512_80k_ade20k_20211201_113849-88b2907c.pth
|
||||
Training log: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-b_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_svt-b_fpn_fpnhead_8x4_512x512_80k_ade20k_20211201_113849.log.json
|
||||
Paper:
|
||||
Title: 'Twins: Revisiting the Design of Spatial Attention in Vision Transformers'
|
||||
URL: https://arxiv.org/pdf/2104.13840.pdf
|
||||
Code: https://github.com/open-mmlab/mmsegmentation/blob/v0.20.0/mmseg/models/backbones/twins.py#L352
|
||||
Framework: PyTorch
|
||||
- Name: twins_svt-b_uperhead_8xb2-160k_ade20k-512x512
|
||||
In Collection: UPerNet
|
||||
Results:
|
||||
Task: Semantic Segmentation
|
||||
Dataset: ADE20K
|
||||
Metrics:
|
||||
mIoU: 48.04
|
||||
mIoU(ms+flip): 48.87
|
||||
Config: configs/twins/twins_svt-b_uperhead_8xb2-160k_ade20k-512x512.py
|
||||
Metadata:
|
||||
Training Data: ADE20K
|
||||
Batch Size: 16
|
||||
Architecture:
|
||||
- Twins-SVT-B
|
||||
- UPerNet
|
||||
Training Resources: 8x V100 GPUS
|
||||
Memory (GB): 6.77
|
||||
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-b_uperhead_8x2_512x512_160k_ade20k/twins_svt-b_uperhead_8x2_512x512_160k_ade20k_20211202_040826-0943a1f1.pth
|
||||
Training log: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-b_uperhead_8x2_512x512_160k_ade20k/twins_svt-b_uperhead_8x2_512x512_160k_ade20k_20211202_040826.log.json
|
||||
Paper:
|
||||
Title: 'Twins: Revisiting the Design of Spatial Attention in Vision Transformers'
|
||||
URL: https://arxiv.org/pdf/2104.13840.pdf
|
||||
Code: https://github.com/open-mmlab/mmsegmentation/blob/v0.20.0/mmseg/models/backbones/twins.py#L352
|
||||
Framework: PyTorch
|
||||
- Name: twins_svt-l_fpn_fpnhead_8xb4-80k_ade20k-512x512
|
||||
In Collection: FPN
|
||||
Results:
|
||||
Task: Semantic Segmentation
|
||||
Dataset: ADE20K
|
||||
Metrics:
|
||||
mIoU: 46.55
|
||||
mIoU(ms+flip): 47.74
|
||||
Config: configs/twins/twins_svt-l_fpn_fpnhead_8xb4-80k_ade20k-512x512.py
|
||||
Metadata:
|
||||
Training Data: ADE20K
|
||||
Batch Size: 32
|
||||
Architecture:
|
||||
- Twins-SVT-L
|
||||
- FPN
|
||||
Training Resources: 8x V100 GPUS
|
||||
Memory (GB): 11.2
|
||||
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-l_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_svt-l_fpn_fpnhead_8x4_512x512_80k_ade20k_20211130_141005-1d59bee2.pth
|
||||
Training log: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-l_fpn_fpnhead_8x4_512x512_80k_ade20k/twins_svt-l_fpn_fpnhead_8x4_512x512_80k_ade20k_20211130_141005.log.json
|
||||
Paper:
|
||||
Title: 'Twins: Revisiting the Design of Spatial Attention in Vision Transformers'
|
||||
URL: https://arxiv.org/pdf/2104.13840.pdf
|
||||
Code: https://github.com/open-mmlab/mmsegmentation/blob/v0.20.0/mmseg/models/backbones/twins.py#L352
|
||||
Framework: PyTorch
|
||||
- Name: twins_pcpvt-l_uperhead_8xb2-160k_ade20k-512x512
|
||||
In Collection: UPerNet
|
||||
Results:
|
||||
Task: Semantic Segmentation
|
||||
Dataset: ADE20K
|
||||
Metrics:
|
||||
mIoU: 49.65
|
||||
mIoU(ms+flip): 50.63
|
||||
Config: configs/twins/twins_pcpvt-l_uperhead_8xb2-160k_ade20k-512x512.py
|
||||
Metadata:
|
||||
Training Data: ADE20K
|
||||
Batch Size: 16
|
||||
Architecture:
|
||||
- Twins-SVT-L
|
||||
- UPerNet
|
||||
Training Resources: 8x V100 GPUS
|
||||
Memory (GB): 8.41
|
||||
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-l_uperhead_8x2_512x512_160k_ade20k/twins_svt-l_uperhead_8x2_512x512_160k_ade20k_20211130_141005-3e2cae61.pth
|
||||
Training log: https://download.openmmlab.com/mmsegmentation/v0.5/twins/twins_svt-l_uperhead_8x2_512x512_160k_ade20k/twins_svt-l_uperhead_8x2_512x512_160k_ade20k_20211130_141005.log.json
|
||||
Paper:
|
||||
Title: 'Twins: Revisiting the Design of Spatial Attention in Vision Transformers'
|
||||
URL: https://arxiv.org/pdf/2104.13840.pdf
|
||||
Code: https://github.com/open-mmlab/mmsegmentation/blob/v0.20.0/mmseg/models/backbones/twins.py#L352
|
||||
Framework: PyTorch
|
||||
@@ -0,0 +1,8 @@
|
||||
_base_ = ['./twins_pcpvt-s_fpn_fpnhead_8xb4-80k_ade20k-512x512.py']
|
||||
|
||||
checkpoint = 'https://download.openmmlab.com/mmsegmentation/v0.5/pretrain/twins/pcpvt_base_20220308-0621964c.pth' # noqa
|
||||
|
||||
model = dict(
|
||||
backbone=dict(
|
||||
init_cfg=dict(type='Pretrained', checkpoint=checkpoint),
|
||||
depths=[3, 4, 18, 3]), )
|
||||
@@ -0,0 +1,13 @@
|
||||
_base_ = ['./twins_pcpvt-s_uperhead_8xb4-160k_ade20k-512x512.py']
|
||||
|
||||
checkpoint = 'https://download.openmmlab.com/mmsegmentation/v0.5/pretrain/twins/pcpvt_base_20220308-0621964c.pth' # noqa
|
||||
|
||||
model = dict(
|
||||
backbone=dict(
|
||||
init_cfg=dict(type='Pretrained', checkpoint=checkpoint),
|
||||
depths=[3, 4, 18, 3],
|
||||
drop_path_rate=0.3))
|
||||
|
||||
train_dataloader = dict(batch_size=2, num_workers=2)
|
||||
val_dataloader = dict(batch_size=1, num_workers=4)
|
||||
test_dataloader = val_dataloader
|
||||
@@ -0,0 +1,8 @@
|
||||
_base_ = ['./twins_pcpvt-s_fpn_fpnhead_8xb4-80k_ade20k-512x512.py']
|
||||
|
||||
checkpoint = 'https://download.openmmlab.com/mmsegmentation/v0.5/pretrain/twins/pcpvt_large_20220308-37579dc6.pth' # noqa
|
||||
|
||||
model = dict(
|
||||
backbone=dict(
|
||||
init_cfg=dict(type='Pretrained', checkpoint=checkpoint),
|
||||
depths=[3, 8, 27, 3]))
|
||||
@@ -0,0 +1,13 @@
|
||||
_base_ = ['./twins_pcpvt-s_uperhead_8xb4-160k_ade20k-512x512.py']
|
||||
|
||||
checkpoint = 'https://download.openmmlab.com/mmsegmentation/v0.5/pretrain/twins/pcpvt_large_20220308-37579dc6.pth' # noqa
|
||||
|
||||
model = dict(
|
||||
backbone=dict(
|
||||
init_cfg=dict(type='Pretrained', checkpoint=checkpoint),
|
||||
depths=[3, 8, 27, 3],
|
||||
drop_path_rate=0.3))
|
||||
|
||||
train_dataloader = dict(batch_size=2, num_workers=2)
|
||||
val_dataloader = dict(batch_size=1, num_workers=4)
|
||||
test_dataloader = val_dataloader
|
||||
@@ -0,0 +1,12 @@
|
||||
_base_ = [
|
||||
'../_base_/models/twins_pcpvt-s_fpn.py', '../_base_/datasets/ade20k.py',
|
||||
'../_base_/default_runtime.py', '../_base_/schedules/schedule_80k.py'
|
||||
]
|
||||
crop_size = (512, 512)
|
||||
data_preprocessor = dict(size=crop_size)
|
||||
model = dict(data_preprocessor=data_preprocessor)
|
||||
optim_wrapper = dict(
|
||||
_delete_=True,
|
||||
type='OptimWrapper',
|
||||
optimizer=dict(type='AdamW', lr=0.0001, weight_decay=0.0001),
|
||||
clip_grad=None)
|
||||
@@ -0,0 +1,31 @@
|
||||
_base_ = [
|
||||
'../_base_/models/twins_pcpvt-s_upernet.py',
|
||||
'../_base_/datasets/ade20k.py', '../_base_/default_runtime.py',
|
||||
'../_base_/schedules/schedule_160k.py'
|
||||
]
|
||||
crop_size = (512, 512)
|
||||
data_preprocessor = dict(size=crop_size)
|
||||
model = dict(data_preprocessor=data_preprocessor)
|
||||
|
||||
optim_wrapper = dict(
|
||||
_delete_=True,
|
||||
type='OptimWrapper',
|
||||
optimizer=dict(
|
||||
type='AdamW', lr=0.00006, betas=(0.9, 0.999), weight_decay=0.01),
|
||||
paramwise_cfg=dict(custom_keys={
|
||||
'pos_block': dict(decay_mult=0.),
|
||||
'norm': dict(decay_mult=0.)
|
||||
}))
|
||||
|
||||
param_scheduler = [
|
||||
dict(
|
||||
type='LinearLR', start_factor=1e-6, by_epoch=False, begin=0, end=1500),
|
||||
dict(
|
||||
type='PolyLR',
|
||||
eta_min=0.0,
|
||||
power=1.0,
|
||||
begin=1500,
|
||||
end=160000,
|
||||
by_epoch=False,
|
||||
)
|
||||
]
|
||||
@@ -0,0 +1,12 @@
|
||||
_base_ = ['./twins_svt-s_fpn_fpnhead_8xb4-80k_ade20k-512x512.py']
|
||||
|
||||
checkpoint = 'https://download.openmmlab.com/mmsegmentation/v0.5/pretrain/twins/alt_gvt_base_20220308-1b7eb711.pth' # noqa
|
||||
|
||||
model = dict(
|
||||
backbone=dict(
|
||||
init_cfg=dict(type='Pretrained', checkpoint=checkpoint),
|
||||
embed_dims=[96, 192, 384, 768],
|
||||
num_heads=[3, 6, 12, 24],
|
||||
depths=[2, 2, 18, 2]),
|
||||
neck=dict(in_channels=[96, 192, 384, 768]),
|
||||
)
|
||||
@@ -0,0 +1,12 @@
|
||||
_base_ = ['./twins_svt-s_uperhead_8xb2-160k_ade20k-512x512.py']
|
||||
|
||||
checkpoint = 'https://download.openmmlab.com/mmsegmentation/v0.5/pretrain/twins/alt_gvt_base_20220308-1b7eb711.pth' # noqa
|
||||
|
||||
model = dict(
|
||||
backbone=dict(
|
||||
init_cfg=dict(type='Pretrained', checkpoint=checkpoint),
|
||||
embed_dims=[96, 192, 384, 768],
|
||||
num_heads=[3, 6, 12, 24],
|
||||
depths=[2, 2, 18, 2]),
|
||||
decode_head=dict(in_channels=[96, 192, 384, 768]),
|
||||
auxiliary_head=dict(in_channels=384))
|
||||
@@ -0,0 +1,13 @@
|
||||
_base_ = ['./twins_svt-s_fpn_fpnhead_8xb4-80k_ade20k-512x512.py']
|
||||
|
||||
checkpoint = 'https://download.openmmlab.com/mmsegmentation/v0.5/pretrain/twins/alt_gvt_large_20220308-fb5936f3.pth' # noqa
|
||||
|
||||
model = dict(
|
||||
backbone=dict(
|
||||
init_cfg=dict(type='Pretrained', checkpoint=checkpoint),
|
||||
embed_dims=[128, 256, 512, 1024],
|
||||
num_heads=[4, 8, 16, 32],
|
||||
depths=[2, 2, 18, 2],
|
||||
drop_path_rate=0.3),
|
||||
neck=dict(in_channels=[128, 256, 512, 1024]),
|
||||
)
|
||||
@@ -0,0 +1,13 @@
|
||||
_base_ = ['./twins_svt-s_uperhead_8xb2-160k_ade20k-512x512.py']
|
||||
|
||||
checkpoint = 'https://download.openmmlab.com/mmsegmentation/v0.5/pretrain/twins/alt_gvt_large_20220308-fb5936f3.pth' # noqa
|
||||
|
||||
model = dict(
|
||||
backbone=dict(
|
||||
init_cfg=dict(type='Pretrained', checkpoint=checkpoint),
|
||||
embed_dims=[128, 256, 512, 1024],
|
||||
num_heads=[4, 8, 16, 32],
|
||||
depths=[2, 2, 18, 2],
|
||||
drop_path_rate=0.3),
|
||||
decode_head=dict(in_channels=[128, 256, 512, 1024]),
|
||||
auxiliary_head=dict(in_channels=512))
|
||||
@@ -0,0 +1,28 @@
|
||||
_base_ = [
|
||||
'../_base_/models/twins_pcpvt-s_fpn.py', '../_base_/datasets/ade20k.py',
|
||||
'../_base_/default_runtime.py', '../_base_/schedules/schedule_80k.py'
|
||||
]
|
||||
crop_size = (512, 512)
|
||||
data_preprocessor = dict(size=crop_size)
|
||||
checkpoint = 'https://download.openmmlab.com/mmsegmentation/v0.5/pretrain/twins/alt_gvt_small_20220308-7e1c3695.pth' # noqa
|
||||
|
||||
model = dict(
|
||||
data_preprocessor=data_preprocessor,
|
||||
backbone=dict(
|
||||
type='SVT',
|
||||
init_cfg=dict(type='Pretrained', checkpoint=checkpoint),
|
||||
embed_dims=[64, 128, 256, 512],
|
||||
num_heads=[2, 4, 8, 16],
|
||||
mlp_ratios=[4, 4, 4, 4],
|
||||
depths=[2, 2, 10, 4],
|
||||
windiow_sizes=[7, 7, 7, 7],
|
||||
norm_after_stage=True),
|
||||
neck=dict(in_channels=[64, 128, 256, 512], out_channels=256, num_outs=4),
|
||||
decode_head=dict(num_classes=150),
|
||||
)
|
||||
|
||||
optim_wrapper = dict(
|
||||
_delete_=True,
|
||||
type='OptimWrapper',
|
||||
optimizer=dict(type='AdamW', lr=0.0001, weight_decay=0.0001),
|
||||
clip_grad=None)
|
||||
@@ -0,0 +1,49 @@
|
||||
_base_ = [
|
||||
'../_base_/models/twins_pcpvt-s_upernet.py',
|
||||
'../_base_/datasets/ade20k.py', '../_base_/default_runtime.py',
|
||||
'../_base_/schedules/schedule_160k.py'
|
||||
]
|
||||
crop_size = (512, 512)
|
||||
data_preprocessor = dict(size=crop_size)
|
||||
checkpoint = 'https://download.openmmlab.com/mmsegmentation/v0.5/pretrain/twins/alt_gvt_small_20220308-7e1c3695.pth' # noqa
|
||||
|
||||
model = dict(
|
||||
data_preprocessor=data_preprocessor,
|
||||
backbone=dict(
|
||||
type='SVT',
|
||||
init_cfg=dict(type='Pretrained', checkpoint=checkpoint),
|
||||
embed_dims=[64, 128, 256, 512],
|
||||
num_heads=[2, 4, 8, 16],
|
||||
mlp_ratios=[4, 4, 4, 4],
|
||||
depths=[2, 2, 10, 4],
|
||||
windiow_sizes=[7, 7, 7, 7],
|
||||
norm_after_stage=True),
|
||||
decode_head=dict(in_channels=[64, 128, 256, 512]),
|
||||
auxiliary_head=dict(in_channels=256))
|
||||
|
||||
optim_wrapper = dict(
|
||||
_delete_=True,
|
||||
type='OptimWrapper',
|
||||
optimizer=dict(
|
||||
type='AdamW', lr=0.00006, betas=(0.9, 0.999), weight_decay=0.01),
|
||||
paramwise_cfg=dict(custom_keys={
|
||||
'pos_block': dict(decay_mult=0.),
|
||||
'norm': dict(decay_mult=0.)
|
||||
}))
|
||||
|
||||
param_scheduler = [
|
||||
dict(
|
||||
type='LinearLR', start_factor=1e-6, by_epoch=False, begin=0, end=1500),
|
||||
dict(
|
||||
type='PolyLR',
|
||||
eta_min=0.0,
|
||||
power=1.0,
|
||||
begin=1500,
|
||||
end=160000,
|
||||
by_epoch=False,
|
||||
)
|
||||
]
|
||||
|
||||
train_dataloader = dict(batch_size=2, num_workers=2)
|
||||
val_dataloader = dict(batch_size=1, num_workers=4)
|
||||
test_dataloader = val_dataloader
|
||||
Reference in New Issue
Block a user