Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition (arxiv)

This is a Pytorch implementation of our paper. We present Vision Permutator, a conceptually simple and data efficient MLP-like architecture for visual recognition. We show that our Vision Permutators are formidable competitors to convolutional neural networks (CNNs) and vision transformers.

We hope this work could encourage researchers to rethink the way of encoding spatial information and facilitate the development of MLP-like models.

Basic structure of the proposed Permute-MLP layer. The proposed Permute-MLP layer contains three branches that are responsible for encoding features along the height, width, and channel dimensions, respectively. The outputs from the three branches are then combined using element-wise addition, followed by a fully-connected layer for feature fusion.

Our code is based on the pytorch-image-models, Token Labeling, T2T-ViT

Comparison with Recent MLP-like Models

Model	Parameters	Throughput	Image resolution	Top 1 Acc.	Download
EAMLP-14	30M	711 img/s	224	78.9%
gMLP-S	20M	-	224	79.6%
ResMLP-S24	30M	715 img/s	224	79.4%
ViP-Small/7 (ours)	25M	719 img/s	224	81.5%	link
EAMLP-19	55M	464 img/s	224	79.4%
Mixer-B/16	59M	-	224	78.5%
ViP-Medium/7 (ours)	55M	418 img/s	224	82.7%	link
gMLP-B	73M	-	224	81.6%
ResMLP-B24	116M	231 img/s	224	81.0%
ViP-Large/7	88M	298 img/s	224	83.2%	link

The throughput is measured on a single machine with V100 GPU (32GB) with batch size set to 32.

Training ViP-Small/7 takes less than 30h on ImageNet for 300 epochs on a node with 8 A100 GPUs.

Requirements

torch>=1.4.0
torchvision>=0.5.0
pyyaml
timm==0.4.5
apex if you use 'apex amp'

data prepare: ImageNet with the following folder structure, you can extract imagenet by this script.

│imagenet/
├──train/
│  ├── n01440764
│  │   ├── n01440764_10026.JPEG
│  │   ├── n01440764_10027.JPEG
│  │   ├── ......
│  ├── ......
├──val/
│  ├── n01440764
│  │   ├── ILSVRC2012_val_00000293.JPEG
│  │   ├── ILSVRC2012_val_00002138.JPEG
│  │   ├── ......
│  ├── ......

Validation

Replace DATA_DIR with your imagenet validation set path and MODEL_DIR with the checkpoint path

CUDA_VISIBLE_DEVICES=0 bash eval.sh /path/to/imagenet/val /path/to/checkpoint

Training

Command line for training on 8 GPUs (V100)

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 ./distributed_train.sh 8 /path/to/imagenet --model vip_s7 -b 256 -j 8 --opt adamw --epochs 300 --sched cosine --apex-amp --img-size 224 --drop-path 0.1 --lr 2e-3 --weight-decay 0.05 --remode pixel --reprob 0.25 --aa rand-m9-mstd0.5-inc1 --smoothing 0.1 --mixup 0.8 --cutmix 1.0 --warmup-lr 1e-6 --warmup-epochs 20

Reference

You may want to cite:

@misc{hou2021vision,
    title={Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition},
    author={Qibin Hou and Zihang Jiang and Li Yuan and Ming-Ming Cheng and Shuicheng Yan and Jiashi Feng},
    year={2021},
    eprint={2106.12368},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

License

This repository is released under the MIT License as found in the LICENSE file. For commercial use, please contact with the authors.

I don't know why,but I got a 84.42%@vip_s7，the following is the cmd and log. Note: I train the vip with 6 Ti1080, batch=64.

CUDA_VISIBLE_DEVICES=0,1,2,4,6,7 ./distributed_train.sh 6 --data /data/path/to/imagenet --model vip_s7 -b 64 -j 6 --opt adamw --epochs 300 --sched cosine --apex-amp --img-size 224 --drop-path 0.1 --lr 2e-3 --weight-decay 0.05 --remode pixel --reprob 0.25 --aa rand-m9-mstd0.5-inc1 --smoothing 0.1 --mixup 0.8 --cutmix 1.0 --warmup-lr 1e-6 --warmup-epochs 20

epoch,train_loss,eval_loss,eval_top1,eval_top5 0,6.897498446451107,6.920823193237253,0.11199104697234573,0.5319574684269399 1,6.635074172221439,6.908653669171921,0.14598833069916906,0.7299416333558549 2,6.299999626589493,6.877433327481934,0.1879849714256746,1.049916047514386 3,5.961820629281058,6.803794402197336,0.42396609998304324,1.7018639201422174 4,5.721933814841257,6.7456617523561375,0.5619550690766516,2.487801056386528 5,5.513945505652629,6.616568194152012,1.1479082055191414,4.049676146414574 6,5.295299509881248,6.326601969986933,1.9978402258825154,6.5834735274629566 7,5.135693771738402,5.957425979278286,3.4817215823600813,10.541156985532588 8,4.9750422289673715,5.509230036514682,6.265498933301202,17.146628777357723 9,4.955138535566733,4.902270209969506,12.071034721885335,28.171747068950303 10,4.849730646106559,4.18796934782815,20.450364618527967,42.05063708428716 11,4.66967850000086,3.578551411771763,29.06567554780554,53.971683791480764 12,4.74806534404486,3.138725295637848,36.47908275797409,62.429007578502606 13,4.681437905405609,2.8611019714461663,41.726663123969125,67.90856960050374 14,4.665561343582583,2.694952508028331,45.61235242660079,71.53227981629634 15,4.644043553043419,2.5987647279492667,48.32413533427946,73.82609615046714 16,4.6149375808071085,2.52884503917955,50.23598260012124,75.46396516855312 17,4.571690149710212,2.461466482797343,51.871851789942745,76.69786689251139 18,4.573833277527715,2.36428716104323,53.73170294325673,78.08775552594464 19,4.600005122977243,2.263626220273777,55.1275915150909,79.41364968878662 20,4.586769611063138,2.160541468110621,56.7774594400551,80.81153736430333 21,4.567850512518009,2.0593713660705912,58.22534356782097,81.88945123861527 22,4.563050458129023,1.9497668935951067,59.70922500577395,82.97336470528647 23,4.49174424292336,1.8379585912881609,61.295098270190145,84.19926647079171 24,4.531819054778193,1.7433091885814114,62.55499736107537,85.25118260009414 25,4.544238956881241,1.6586213802473133,63.780899418610474,86.10711376244197 26,4.501337981559861,1.5895183448571795,64.77482004442574,86.7730603658641 27,4.434042557864122,1.5319257438225742,65.63675117538449,87.31701739983848 28,4.5165048317170475,1.4855170154388442,66.23270318672358,87.74098337208707 29,4.477951758344409,1.4469930333200907,66.78065965294066,88.03095997864604 30,4.414669812565118,1.4166177173927588,67.1686282587017,88.26894115377507 31,4.408424125590795,1.3925092366206895,67.47260409474859,88.53891952803932 32,4.4501715982464,1.3740543007764823,67.75258193944094,88.7189051877803 33,4.436891948673087,1.3572717362924076,68.01856057203901,88.88689186603354 34,4.47835459507687,1.343482419458659,68.24654237377807,89.01488126128132 35,4.398498310169703,1.3305243780380076,68.3825315617817,89.14287107946892 36,4.421788746202496,1.3195938538784084,68.58451535302957,89.26686126950665 37,4.420958277205346,1.3100458558926857,68.7265037498172,89.35885389056571 38,4.417498296415302,1.3024063418173006,68.84249466707702,89.50484254350472 39,4.377080974444537,1.2944942471618872,69.00448140418298,89.6288324094717 40,4.384807224005041,1.2872051226883905,69.17446816657011,89.7348238325016 41,4.350281732183107,1.2810902684204637,69.31245737279494,89.79081968486466 42,4.38020530888732,1.2767441393461105,69.42244829764205,89.86481388129575 43,4.358453169674941,1.271780503977681,69.59243459131667,89.88881151470773 44,4.372297901502797,1.2678588287476187,69.70842538041293,90.01480166987032 45,4.352815063906387,1.2648489257257736,69.81641682898767,90.08479558085548 46,4.3347624590699105,1.2628966046661274,69.8984102414385,90.13679155789607 47,4.372627782150054,1.2602339647987464,69.94640632484753,90.16278951712184 48,4.336176442428374,1.2586284394970264,70.08239564101473,90.2187850362596 49,4.3249931939890685,1.256730695808709,70.1683887759072,90.28677945998695 50,4.387032535714163,1.2546128732339352,70.29237887623043,90.33077638945134 51,4.292173899395365,1.252888638251744,70.30837772846337,90.36677337419452 52,4.334522573041244,1.25077576496517,70.4223681404804,90.40077038099648 53,4.343562552626704,1.2486651011843002,70.46036552061187,90.44876675735082 54,4.2876138149852485,1.2462088407187373,70.55435769209508,90.48276395822904 55,4.33589161953456,1.2429493092105977,70.65035050613919,90.53475971922246 56,4.266452463579849,1.2397431261338632,70.74834232957409,90.55075857603266 57,4.326588875810865,1.2369515137733074,70.7603412076155,90.5827559381187 58,4.357701543351294,1.233855538497533,70.79433852292549,90.64675089664702 59,4.318143364409326,1.2319323991968216,70.93032786380996,90.68474789594957 60,4.314157845268787,1.2295630219058793,71.00432144505588,90.72874470823584 61,4.297892332077026,1.2265230200689474,71.10431390464349,90.81073740663247 62,4.281018173190909,1.224780632947771,71.13831120439076,90.83273619360452 63,4.24098925523355,1.2214524747657638,71.30029815754388,90.85073457795251 64,4.301690471004433,1.2191791983568498,71.33829531430455,90.87873241625044 65,4.255715172055742,1.2171746795343958,71.38829123284395,90.89073145632725 66,4.281312667148214,1.2140680893107745,71.46028553189377,90.91072970143834 67,4.293928156436329,1.2116449428514409,71.60027411918375,90.98672405157782 68,4.277459722169688,1.2099077029854153,71.65027039291935,91.02072099979073 69,4.311910683000591,1.2092339148588938,71.73626323028924,91.06071793675451 70,4.237875575750646,1.2071373786232813,71.79225863224887,91.06271781582669 71,4.29129040073341,1.2068184426799506,71.79425837583383,91.08471543101747 72,4.292811004208847,1.2045458831002107,71.86825275901757,91.17670852078903 73,4.2103998963262,1.202459813423056,71.91024979842433,91.21070546900194 74,4.268061406175855,1.2004286657857013,72.01824037548676,91.2327035712143 75,4.276580367289799,1.1972193041092662,72.03023983667244,91.25670206149132 76,4.1927781541582565,1.194762274204526,72.12423260045446,91.30069840506509 77,4.223329272068722,1.192597238400127,72.1082336850552,91.32469660239681 78,4.239088642765099,1.1889550637821227,72.18622731189805,91.35469416230882 79,4.239531100635797,1.1856474060505335,72.25822169700056,91.38669133031857 80,4.253520881625968,1.182930325586389,72.36021350835878,91.41668928204494 81,4.264392399452102,1.180294969627146,72.42020867944822,91.46668532142428 82,4.2935264076985105,1.1780071623582247,72.4242081336618,91.54467927599971 83,4.183295256654981,1.1759697122770676,72.55419775402343,91.54667892071564 84,4.1724416101482555,1.173063437430041,72.67818824616101,91.58467580284007 85,4.253374935875477,1.1703134040223742,72.76418096635277,91.63467209488475 86,4.173966360763765,1.1681256303691108,72.82817604205921,91.6386715600838 87,4.203923020564335,1.165260956960815,72.89217057947864,91.67066915652607 88,4.198879456855882,1.1630617950584552,72.9921627168264,91.68866748228501 89,4.204212742792049,1.1595675451795002,73.05015833108351,91.76666124278417 90,4.220558797809439,1.1575091804049413,73.14615044663614,91.78265986523813 91,4.234742809349383,1.1546186548301234,73.23614321791757,91.81865680970132 92,4.211457870375942,1.151942806447585,73.24414290997345,91.7766602280216 93,4.176632179340846,1.1492926530595418,73.28013967866946,91.8186568499813 94,4.13435952428361,1.1468784104785046,73.32213649470542,91.85265428521578 95,4.150933070921562,1.1448545083457227,73.37213264027746,91.85665412025827 96,4.212816624574258,1.1424470642415745,73.41812883545977,91.89265110500145 97,4.1783625945239,1.1403365648991488,73.47612398100438,91.89665058850956 98,4.149515078101359,1.1381833895179103,73.51612139766611,91.95864578514383 99,4.184193624577052,1.135834064362345,73.52012092145421,91.99664300049355 100,4.1438532412891655,1.1334068423871488,73.57611648087196,92.0086419417013 101,4.2061415289489315,1.131613763940114,73.59811418394634,92.05263816809695 102,4.1770783679586065,1.128588039811521,73.66810895545837,92.07463654494555 103,4.130896655606552,1.1262294837030713,73.7041060628724,92.14463090633413 104,4.164088279428617,1.1236643548031233,73.79009931036572,92.19462712148068 105,4.1543260863129525,1.1212902866967267,73.89409091291252,92.23062394876574 106,4.121105573546719,1.118689010662228,73.96008545756769,92.2386235236435 107,4.152086059812089,1.1163591871680416,74.07007676690553,92.25462214609745 108,4.112779476273228,1.1132477346529686,74.08007607621371,92.25062268089842 109,4.138080627145902,1.1113340086521564,74.17006893537872,92.30661806454897 110,4.109547010609801,1.1093562793067986,74.19806683226572,92.35461428344522 111,4.1400179493595175,1.1067077219271677,74.17206843362199,92.43460804019468 112,4.122481537536836,1.1039926298293445,74.2740603328638,92.42660852390598 113,4.130142235420119,1.1019119612455996,74.35605391009638,92.4746044498569 114,4.106339105418031,1.0986333671793653,74.43204776222882,92.53659964649117 115,4.10345072813437,1.0962525172172932,74.53603968518456,92.54259916652957 116,4.145501025965516,1.0950553909262393,74.63003195095955,92.56259748853881 117,4.055538654327393,1.092459419948293,74.63803127317198,92.5325999066559 118,4.107625719527124,1.0902880965476132,74.69802623919969,92.55659773414416 119,4.097143169859765,1.0863236726431758,74.75002239200747,92.68258761467051 120,4.103760034265653,1.0848064310464294,74.78002005078851,92.7085856910744 121,4.094524071250163,1.0829915166444735,74.83401578972315,92.68458761092081 122,4.0647215876780765,1.0807031207671691,74.87601228351927,92.76458164230651 123,4.111846302596616,1.0778105026073013,74.94800669974722,92.8305760111945 124,4.051295757293701,1.075635140280315,75.0020022043256,92.87657223384043 125,4.074876647599986,1.073448331057954,75.04999838294187,92.89057127016753 126,4.065043056514901,1.0720190838254668,75.08399538974382,92.92056875318139 127,4.084583755949853,1.0708788350700102,75.1779882999985,92.92856826947008 128,4.036718828577391,1.0690497463488138,75.17998821935066,93.00056245134178 129,3.981601013264186,1.066288032891054,75.28198010760701,92.9965627517865 130,4.079461373073954,1.0648121422133001,75.35997348727727,92.99456281412526 131,4.064721067186812,1.0631430921492773,75.3959705965222,93.04255920878869 132,4.068798142419735,1.0618841971943698,75.44196721830613,93.05255825261519 133,4.034127584645446,1.0586237160170253,75.53795944005144,93.13055220719062 134,3.9986095159826145,1.056434438444311,75.58195622304318,93.15255005673765 135,4.0494951160860735,1.0536063259844304,75.6919472870393,93.19654663466767 136,4.010537627717139,1.0510139629541613,75.77594024716792,93.2345438280465 137,3.952524077724403,1.0484931318771933,75.75594183544418,93.2405430148596 138,3.992410169520848,1.0460639876486273,75.90992977214655,93.26054180558134 139,3.979717647525626,1.0429913537348379,75.91992850288774,93.31453719298159 140,4.027233473012145,1.0405188840672013,75.97392467574763,93.32053665443092 141,4.0813811396209285,1.0375410683698039,76.02792021511326,93.36253317752158 142,3.9673641164537887,1.034782726930033,76.09991454528853,93.37853226868803 143,3.9360495486729583,1.0329528738618727,76.12391277008386,93.42452866710116 144,3.982970573532749,1.031459635815118,76.2259044075058,93.424528630483 145,4.010461907991221,1.0283353227432723,76.29989843915514,93.5025224678803 146,3.9713859087984327,1.0256409141866887,76.33789559042307,93.49452330312599 147,3.9882485933706793,1.0228629822864903,76.37789218683794,93.5425190716188 148,3.9469488439425615,1.0196745478623428,76.46388505350237,93.57651641164605 149,3.972708628211223,1.0163749271312261,76.5638773849264,93.63051203340257 150,3.9860113741646352,1.012866180578753,76.58387576735561,93.69050676507403 151,3.9316770627465045,1.009717572527822,76.68186806591117,93.74450299103025 152,3.995447256195713,1.006326335381054,76.79585906381884,93.76850124695102 153,3.9208318649883003,1.0038157002028156,76.81585723935542,93.82449610330002 154,3.966935953623812,1.0022543991730601,76.8718528280677,93.82249634140597 155,3.900635373424476,1.000155895025804,77.0498383633948,93.89249083828176 156,3.9117869793529243,0.998312960948079,77.1458308304818,93.93648741621178 157,3.9204770645625153,0.9957889344499413,77.173828503998,93.97248465362028 158,3.9169989001583048,0.9928011367267804,77.20982563155202,94.00848159808348 159,3.9202801711122754,0.9887251343863286,77.2998185493061,94.0104815540538 160,3.906469684251597,0.9861672291917025,77.37181281906139,94.03447967448737 161,3.9593827623716544,0.983605310446014,77.42980836374399,94.06847699254372 162,3.876635793229224,0.9816954550709918,77.46580536679625,94.10447421164315 163,3.9001552756403535,0.9796518905288114,77.49780249452603,94.14847049662785 164,3.835129989704616,0.9770739230143815,77.57379640524753,94.1584697748106 165,3.898776393541148,0.9746265718626238,77.61979292083879,94.1744686316208 166,3.916073674886999,0.9713456796346764,77.66978896021814,94.23046389809323 167,3.854554978894516,0.968392954846495,77.72378455268012,94.29245927049469 168,3.8261295976773115,0.9656309809420416,77.78577933919095,94.32845609777975 169,3.8342656820592746,0.9632433774175477,77.92376818289596,94.40045051950044 170,3.8310181825933323,0.9598682177905016,78.00576187730665,94.46644512274467 171,3.8278482430417773,0.9563763968401571,78.08975523474236,94.50044233643926 172,3.8315204096512057,0.9536149256344405,78.19574680424492,94.51444107982104 173,3.815767133739633,0.9502094194761477,78.2917392420374,94.55643795444607 174,3.8109556050367757,0.9475126778561633,78.31773717196863,94.60443399757511 175,3.798724056969226,0.944900313612652,78.37973222213024,94.61043386914788 176,3.8188754739895674,0.9417507125228779,78.465725520889,94.61243374822007 177,3.7906133960670148,0.9377580324694325,78.4777239640897,94.61243348273837 178,3.841992200260431,0.9353244338700432,78.59571483731985,94.62443275717142 179,3.8163234885309785,0.9324265027864391,78.68970713238937,94.68842724698723 180,3.8143192613628547,0.9294543408191773,78.73970324134324,94.73642334870534 181,3.7683921229671427,0.9261675824185255,78.8036984342278,94.78241980570753 182,3.7818730481913394,0.923982460521124,78.83169595028589,94.84441482657462 183,3.755684365688915,0.9212717203776957,78.88969158285208,94.85041461209471 184,3.811672761406697,0.9191701354698967,78.96368532155611,94.87241293035424 185,3.7187630223556303,0.9165008361030031,79.07167678843993,94.91240963296178 186,3.7433232388026276,0.9128048724913043,79.16366926302634,94.94440711222595 187,3.783568885964407,0.9098437855447944,79.193666969411,94.97040483709546 188,3.7836115662480743,0.9066717340873534,79.25566220632525,94.9604057346799 189,3.7132954631053225,0.9037563167718037,79.32965616840008,94.99040315910469 190,3.7004594366315384,0.9007076963349615,79.47364497156146,95.0224008727251 191,3.7304701771534665,0.8978327910105387,79.55763772479744,95.08239607310907 192,3.700018348828168,0.8942069873439056,79.58763541287303,95.15439033187891 193,3.6864709585485325,0.8912874848430095,79.66762916962249,95.17638853296033 194,3.716994675112442,0.8881717415373188,79.77762017502955,95.21238559460166 195,3.6630856252052415,0.8846277142989999,79.83161617761498,95.24038335600144 196,3.70445132591355,0.8816261793859348,79.90161046942904,95.2903792196136 197,3.667574173967603,0.8784933531916568,79.9416069962694,95.34837442374727 198,3.673860294718138,0.8758927606294731,80.07559646265896,95.36637298386246 199,3.647094535156035,0.8726708035918772,80.16358933655913,95.41636914041993 200,3.6449320484215106,0.8685720142827263,80.24158311536738,95.41236961663184 201,3.6660440975511577,0.8651743567416157,80.33757523092001,95.43236791667019 202,3.6045285580863413,0.8614645302081125,80.4675654142859,95.4943630547154 203,3.6495057394806767,0.8582117335241934,80.51956098120304,95.52636053397954 204,3.6250970833738085,0.854794814142072,80.60955384036806,95.57435679315577 205,3.6242427893087896,0.8516537740072072,80.68154798195978,95.60035451802527 206,3.6249839084249147,0.8486535416422972,80.76154147505846,95.64635107389651 207,3.6208607680361036,0.8452714143509998,80.81553724328762,95.69034761154654 208,3.621414886394017,0.8423744141697912,80.89553079497537,95.71434561480199 209,3.595421079178931,0.8396461595297718,80.90353028196954,95.77434071631691 210,3.600605377009217,0.8370815479186103,81.0335199133166,95.81833705989068 211,3.6281545095040766,0.83375701421967,81.12551273943738,95.85633405919323 212,3.604720505190567,0.8295639896667458,81.24350308536596,95.89833075805107 213,3.5827912982081025,0.8255833837377787,81.4014905623346,95.91232946115288 214,3.567787086459952,0.8213720157159613,81.44348708542526,95.95632580472663 215,3.5835306644439697,0.817864267208721,81.5234805831012,96.00632172692787 216,3.5531847208318577,0.8139958393382087,81.59347526215237,96.02032093902214 217,3.556191263064532,0.8101979788756639,81.68346796111291,96.10231420500031 218,3.5177630505091706,0.8064951221806156,81.75546192693744,96.16030967461568 219,3.505100737155323,0.8028557199295516,81.83345549336033,96.17230883736332 220,3.548047834718731,0.7992166765078899,81.90345040035957,96.20830599787368 221,3.5126069230093084,0.7949462399827166,82.04543895002804,96.22230474125547 222,3.4366250877649014,0.791103566382353,82.14943049398578,96.26030164168898 223,3.5024204757851614,0.7875747270998349,82.24342289890978,96.31829690441171 224,3.52920873735992,0.7836198607651427,82.2774204330133,96.35229432133711 225,3.5523327135703933,0.7802904253530079,82.40141039784932,96.3882917345128 226,3.514156509453142,0.7771999123058131,82.44140739340219,96.42828861288753 227,3.480050802230835,0.7738232632063263,82.5373997726056,96.46428532299447 228,3.490331743804502,0.769954922446994,82.67138911083158,96.4842838207709 229,3.4522565519306023,0.7658971668519875,82.7413832342021,96.54027943877772 230,3.4249578193879464,0.7622650279282437,82.76738133532325,96.57427603545622 231,3.437175576115998,0.7588821934200403,82.83737553925373,96.60227404952136 232,3.4281330276542987,0.7550330743933857,82.89737097399396,96.63827128692986 233,3.412930045329349,0.7511200292064099,82.97936447432838,96.69026726397044 234,3.42685407316181,0.7473738250727845,83.14735111230164,96.70226630404724 235,3.4242665700509516,0.7433231376417255,83.16534943806057,96.74026295181542 236,3.406816643728337,0.7396762944652446,83.29333929652816,96.76226162160934 237,3.379652134129699,0.7358025643989741,83.37533269799354,96.80825790284433 238,3.421702680453448,0.7321584496476365,83.42132862769417,96.87225303713984 239,3.4100404222246627,0.7282632348175269,83.57931626212091,96.92824830361228 240,3.3646818852760423,0.7243552193070648,83.67330864507402,96.9402472851 241,3.376362484945378,0.7204217895032234,83.767300910849,96.99224302778434 242,3.3655178748386008,0.7166998286827423,83.88929142504537,97.04423923918118 243,3.366966569927377,0.7125734683561589,84.01128154742737,97.06023780304606 244,3.359389214448526,0.7084448325539596,84.1212729226779,97.10023482972436 245,3.358978704667427,0.7042798031884073,84.22926440054718,97.12023303455548 246,3.30417195172377,0.6996296609595207,84.33325582732678,97.1522305724087 247,3.296816946754993,0.6952631592550294,84.42524891709834,97.20822616295189

questions regarding training loss and parser

Hi, thanks for your work and sharing the logs. I'm training the model with some data but I can see my training loss is way lower than your loss on the log(like first five epochs drops to 1.0). I'm thinking it might be the parser's faults or somewhere else. How do you configure the parser? I'm new to timm and I see your code doesn't configure parser but leaves timm to auto configure it. Any help would be appreciated. Thanks.

opened by chengengliu 3
Missing Citation?

The paper looks to be heavily inspired by our WACV paper - Rotate to Attend: Convolutional Triplet Attention Module. Considering there is a mutual author in both the papers, we find it surprising that our paper wasn't at least mentioned in the literature review of this paper. We believe it is important and crucial in the academic space to ensure appropriate and relevant papers are cited.

opened by digantamisra98 3
Question about pre-training for cifar10 and cifar-100

Hi! Thank you for releasing the code.

Could you explain the pre-training done for cifar-10 and cifar-100 in transfer_learning.py? I found no details about this in the paper, too.

opened by sairaamVenkatraman 0
A higher result than paper, but i don't know why.

I don't know why,but I got a 84.42%@vip_s7，the following is the cmd and log. Note: I train the vip with 6 Ti1080, batch=64.

CUDA_VISIBLE_DEVICES=0,1,2,4,6,7 ./distributed_train.sh 6 --data /data/path/to/imagenet --model vip_s7 -b 64 -j 6 --opt adamw --epochs 300 --sched cosine --apex-amp --img-size 224 --drop-path 0.1 --lr 2e-3 --weight-decay 0.05 --remode pixel --reprob 0.25 --aa rand-m9-mstd0.5-inc1 --smoothing 0.1 --mixup 0.8 --cutmix 1.0 --warmup-lr 1e-6 --warmup-epochs 20

epoch,train_loss,eval_loss,eval_top1,eval_top5 0,6.897498446451107,6.920823193237253,0.11199104697234573,0.5319574684269399 1,6.635074172221439,6.908653669171921,0.14598833069916906,0.7299416333558549 2,6.299999626589493,6.877433327481934,0.1879849714256746,1.049916047514386 3,5.961820629281058,6.803794402197336,0.42396609998304324,1.7018639201422174 4,5.721933814841257,6.7456617523561375,0.5619550690766516,2.487801056386528 5,5.513945505652629,6.616568194152012,1.1479082055191414,4.049676146414574 6,5.295299509881248,6.326601969986933,1.9978402258825154,6.5834735274629566 7,5.135693771738402,5.957425979278286,3.4817215823600813,10.541156985532588 8,4.9750422289673715,5.509230036514682,6.265498933301202,17.146628777357723 9,4.955138535566733,4.902270209969506,12.071034721885335,28.171747068950303 10,4.849730646106559,4.18796934782815,20.450364618527967,42.05063708428716 11,4.66967850000086,3.578551411771763,29.06567554780554,53.971683791480764 12,4.74806534404486,3.138725295637848,36.47908275797409,62.429007578502606 13,4.681437905405609,2.8611019714461663,41.726663123969125,67.90856960050374 14,4.665561343582583,2.694952508028331,45.61235242660079,71.53227981629634 15,4.644043553043419,2.5987647279492667,48.32413533427946,73.82609615046714 16,4.6149375808071085,2.52884503917955,50.23598260012124,75.46396516855312 17,4.571690149710212,2.461466482797343,51.871851789942745,76.69786689251139 18,4.573833277527715,2.36428716104323,53.73170294325673,78.08775552594464 19,4.600005122977243,2.263626220273777,55.1275915150909,79.41364968878662 20,4.586769611063138,2.160541468110621,56.7774594400551,80.81153736430333 21,4.567850512518009,2.0593713660705912,58.22534356782097,81.88945123861527 22,4.563050458129023,1.9497668935951067,59.70922500577395,82.97336470528647 23,4.49174424292336,1.8379585912881609,61.295098270190145,84.19926647079171 24,4.531819054778193,1.7433091885814114,62.55499736107537,85.25118260009414 25,4.544238956881241,1.6586213802473133,63.780899418610474,86.10711376244197 26,4.501337981559861,1.5895183448571795,64.77482004442574,86.7730603658641 27,4.434042557864122,1.5319257438225742,65.63675117538449,87.31701739983848 28,4.5165048317170475,1.4855170154388442,66.23270318672358,87.74098337208707 29,4.477951758344409,1.4469930333200907,66.78065965294066,88.03095997864604 30,4.414669812565118,1.4166177173927588,67.1686282587017,88.26894115377507 31,4.408424125590795,1.3925092366206895,67.47260409474859,88.53891952803932 32,4.4501715982464,1.3740543007764823,67.75258193944094,88.7189051877803 33,4.436891948673087,1.3572717362924076,68.01856057203901,88.88689186603354 34,4.47835459507687,1.343482419458659,68.24654237377807,89.01488126128132 35,4.398498310169703,1.3305243780380076,68.3825315617817,89.14287107946892 36,4.421788746202496,1.3195938538784084,68.58451535302957,89.26686126950665 37,4.420958277205346,1.3100458558926857,68.7265037498172,89.35885389056571 38,4.417498296415302,1.3024063418173006,68.84249466707702,89.50484254350472 39,4.377080974444537,1.2944942471618872,69.00448140418298,89.6288324094717 40,4.384807224005041,1.2872051226883905,69.17446816657011,89.7348238325016 41,4.350281732183107,1.2810902684204637,69.31245737279494,89.79081968486466 42,4.38020530888732,1.2767441393461105,69.42244829764205,89.86481388129575 43,4.358453169674941,1.271780503977681,69.59243459131667,89.88881151470773 44,4.372297901502797,1.2678588287476187,69.70842538041293,90.01480166987032 45,4.352815063906387,1.2648489257257736,69.81641682898767,90.08479558085548 46,4.3347624590699105,1.2628966046661274,69.8984102414385,90.13679155789607 47,4.372627782150054,1.2602339647987464,69.94640632484753,90.16278951712184 48,4.336176442428374,1.2586284394970264,70.08239564101473,90.2187850362596 49,4.3249931939890685,1.256730695808709,70.1683887759072,90.28677945998695 50,4.387032535714163,1.2546128732339352,70.29237887623043,90.33077638945134 51,4.292173899395365,1.252888638251744,70.30837772846337,90.36677337419452 52,4.334522573041244,1.25077576496517,70.4223681404804,90.40077038099648 53,4.343562552626704,1.2486651011843002,70.46036552061187,90.44876675735082 54,4.2876138149852485,1.2462088407187373,70.55435769209508,90.48276395822904 55,4.33589161953456,1.2429493092105977,70.65035050613919,90.53475971922246 56,4.266452463579849,1.2397431261338632,70.74834232957409,90.55075857603266 57,4.326588875810865,1.2369515137733074,70.7603412076155,90.5827559381187 58,4.357701543351294,1.233855538497533,70.79433852292549,90.64675089664702 59,4.318143364409326,1.2319323991968216,70.93032786380996,90.68474789594957 60,4.314157845268787,1.2295630219058793,71.00432144505588,90.72874470823584 61,4.297892332077026,1.2265230200689474,71.10431390464349,90.81073740663247 62,4.281018173190909,1.224780632947771,71.13831120439076,90.83273619360452 63,4.24098925523355,1.2214524747657638,71.30029815754388,90.85073457795251 64,4.301690471004433,1.2191791983568498,71.33829531430455,90.87873241625044 65,4.255715172055742,1.2171746795343958,71.38829123284395,90.89073145632725 66,4.281312667148214,1.2140680893107745,71.46028553189377,90.91072970143834 67,4.293928156436329,1.2116449428514409,71.60027411918375,90.98672405157782 68,4.277459722169688,1.2099077029854153,71.65027039291935,91.02072099979073 69,4.311910683000591,1.2092339148588938,71.73626323028924,91.06071793675451 70,4.237875575750646,1.2071373786232813,71.79225863224887,91.06271781582669 71,4.29129040073341,1.2068184426799506,71.79425837583383,91.08471543101747 72,4.292811004208847,1.2045458831002107,71.86825275901757,91.17670852078903 73,4.2103998963262,1.202459813423056,71.91024979842433,91.21070546900194 74,4.268061406175855,1.2004286657857013,72.01824037548676,91.2327035712143 75,4.276580367289799,1.1972193041092662,72.03023983667244,91.25670206149132 76,4.1927781541582565,1.194762274204526,72.12423260045446,91.30069840506509 77,4.223329272068722,1.192597238400127,72.1082336850552,91.32469660239681 78,4.239088642765099,1.1889550637821227,72.18622731189805,91.35469416230882 79,4.239531100635797,1.1856474060505335,72.25822169700056,91.38669133031857 80,4.253520881625968,1.182930325586389,72.36021350835878,91.41668928204494 81,4.264392399452102,1.180294969627146,72.42020867944822,91.46668532142428 82,4.2935264076985105,1.1780071623582247,72.4242081336618,91.54467927599971 83,4.183295256654981,1.1759697122770676,72.55419775402343,91.54667892071564 84,4.1724416101482555,1.173063437430041,72.67818824616101,91.58467580284007 85,4.253374935875477,1.1703134040223742,72.76418096635277,91.63467209488475 86,4.173966360763765,1.1681256303691108,72.82817604205921,91.6386715600838 87,4.203923020564335,1.165260956960815,72.89217057947864,91.67066915652607 88,4.198879456855882,1.1630617950584552,72.9921627168264,91.68866748228501 89,4.204212742792049,1.1595675451795002,73.05015833108351,91.76666124278417 90,4.220558797809439,1.1575091804049413,73.14615044663614,91.78265986523813 91,4.234742809349383,1.1546186548301234,73.23614321791757,91.81865680970132 92,4.211457870375942,1.151942806447585,73.24414290997345,91.7766602280216 93,4.176632179340846,1.1492926530595418,73.28013967866946,91.8186568499813 94,4.13435952428361,1.1468784104785046,73.32213649470542,91.85265428521578 95,4.150933070921562,1.1448545083457227,73.37213264027746,91.85665412025827 96,4.212816624574258,1.1424470642415745,73.41812883545977,91.89265110500145 97,4.1783625945239,1.1403365648991488,73.47612398100438,91.89665058850956 98,4.149515078101359,1.1381833895179103,73.51612139766611,91.95864578514383 99,4.184193624577052,1.135834064362345,73.52012092145421,91.99664300049355 100,4.1438532412891655,1.1334068423871488,73.57611648087196,92.0086419417013 101,4.2061415289489315,1.131613763940114,73.59811418394634,92.05263816809695 102,4.1770783679586065,1.128588039811521,73.66810895545837,92.07463654494555 103,4.130896655606552,1.1262294837030713,73.7041060628724,92.14463090633413 104,4.164088279428617,1.1236643548031233,73.79009931036572,92.19462712148068 105,4.1543260863129525,1.1212902866967267,73.89409091291252,92.23062394876574 106,4.121105573546719,1.118689010662228,73.96008545756769,92.2386235236435 107,4.152086059812089,1.1163591871680416,74.07007676690553,92.25462214609745 108,4.112779476273228,1.1132477346529686,74.08007607621371,92.25062268089842 109,4.138080627145902,1.1113340086521564,74.17006893537872,92.30661806454897 110,4.109547010609801,1.1093562793067986,74.19806683226572,92.35461428344522 111,4.1400179493595175,1.1067077219271677,74.17206843362199,92.43460804019468 112,4.122481537536836,1.1039926298293445,74.2740603328638,92.42660852390598 113,4.130142235420119,1.1019119612455996,74.35605391009638,92.4746044498569 114,4.106339105418031,1.0986333671793653,74.43204776222882,92.53659964649117 115,4.10345072813437,1.0962525172172932,74.53603968518456,92.54259916652957 116,4.145501025965516,1.0950553909262393,74.63003195095955,92.56259748853881 117,4.055538654327393,1.092459419948293,74.63803127317198,92.5325999066559 118,4.107625719527124,1.0902880965476132,74.69802623919969,92.55659773414416 119,4.097143169859765,1.0863236726431758,74.75002239200747,92.68258761467051 120,4.103760034265653,1.0848064310464294,74.78002005078851,92.7085856910744 121,4.094524071250163,1.0829915166444735,74.83401578972315,92.68458761092081 122,4.0647215876780765,1.0807031207671691,74.87601228351927,92.76458164230651 123,4.111846302596616,1.0778105026073013,74.94800669974722,92.8305760111945 124,4.051295757293701,1.075635140280315,75.0020022043256,92.87657223384043 125,4.074876647599986,1.073448331057954,75.04999838294187,92.89057127016753 126,4.065043056514901,1.0720190838254668,75.08399538974382,92.92056875318139 127,4.084583755949853,1.0708788350700102,75.1779882999985,92.92856826947008 128,4.036718828577391,1.0690497463488138,75.17998821935066,93.00056245134178 129,3.981601013264186,1.066288032891054,75.28198010760701,92.9965627517865 130,4.079461373073954,1.0648121422133001,75.35997348727727,92.99456281412526 131,4.064721067186812,1.0631430921492773,75.3959705965222,93.04255920878869 132,4.068798142419735,1.0618841971943698,75.44196721830613,93.05255825261519 133,4.034127584645446,1.0586237160170253,75.53795944005144,93.13055220719062 134,3.9986095159826145,1.056434438444311,75.58195622304318,93.15255005673765 135,4.0494951160860735,1.0536063259844304,75.6919472870393,93.19654663466767 136,4.010537627717139,1.0510139629541613,75.77594024716792,93.2345438280465 137,3.952524077724403,1.0484931318771933,75.75594183544418,93.2405430148596 138,3.992410169520848,1.0460639876486273,75.90992977214655,93.26054180558134 139,3.979717647525626,1.0429913537348379,75.91992850288774,93.31453719298159 140,4.027233473012145,1.0405188840672013,75.97392467574763,93.32053665443092 141,4.0813811396209285,1.0375410683698039,76.02792021511326,93.36253317752158 142,3.9673641164537887,1.034782726930033,76.09991454528853,93.37853226868803 143,3.9360495486729583,1.0329528738618727,76.12391277008386,93.42452866710116 144,3.982970573532749,1.031459635815118,76.2259044075058,93.424528630483 145,4.010461907991221,1.0283353227432723,76.29989843915514,93.5025224678803 146,3.9713859087984327,1.0256409141866887,76.33789559042307,93.49452330312599 147,3.9882485933706793,1.0228629822864903,76.37789218683794,93.5425190716188 148,3.9469488439425615,1.0196745478623428,76.46388505350237,93.57651641164605 149,3.972708628211223,1.0163749271312261,76.5638773849264,93.63051203340257 150,3.9860113741646352,1.012866180578753,76.58387576735561,93.69050676507403 151,3.9316770627465045,1.009717572527822,76.68186806591117,93.74450299103025 152,3.995447256195713,1.006326335381054,76.79585906381884,93.76850124695102 153,3.9208318649883003,1.0038157002028156,76.81585723935542,93.82449610330002 154,3.966935953623812,1.0022543991730601,76.8718528280677,93.82249634140597 155,3.900635373424476,1.000155895025804,77.0498383633948,93.89249083828176 156,3.9117869793529243,0.998312960948079,77.1458308304818,93.93648741621178 157,3.9204770645625153,0.9957889344499413,77.173828503998,93.97248465362028 158,3.9169989001583048,0.9928011367267804,77.20982563155202,94.00848159808348 159,3.9202801711122754,0.9887251343863286,77.2998185493061,94.0104815540538 160,3.906469684251597,0.9861672291917025,77.37181281906139,94.03447967448737 161,3.9593827623716544,0.983605310446014,77.42980836374399,94.06847699254372 162,3.876635793229224,0.9816954550709918,77.46580536679625,94.10447421164315 163,3.9001552756403535,0.9796518905288114,77.49780249452603,94.14847049662785 164,3.835129989704616,0.9770739230143815,77.57379640524753,94.1584697748106 165,3.898776393541148,0.9746265718626238,77.61979292083879,94.1744686316208 166,3.916073674886999,0.9713456796346764,77.66978896021814,94.23046389809323 167,3.854554978894516,0.968392954846495,77.72378455268012,94.29245927049469 168,3.8261295976773115,0.9656309809420416,77.78577933919095,94.32845609777975 169,3.8342656820592746,0.9632433774175477,77.92376818289596,94.40045051950044 170,3.8310181825933323,0.9598682177905016,78.00576187730665,94.46644512274467 171,3.8278482430417773,0.9563763968401571,78.08975523474236,94.50044233643926 172,3.8315204096512057,0.9536149256344405,78.19574680424492,94.51444107982104 173,3.815767133739633,0.9502094194761477,78.2917392420374,94.55643795444607 174,3.8109556050367757,0.9475126778561633,78.31773717196863,94.60443399757511 175,3.798724056969226,0.944900313612652,78.37973222213024,94.61043386914788 176,3.8188754739895674,0.9417507125228779,78.465725520889,94.61243374822007 177,3.7906133960670148,0.9377580324694325,78.4777239640897,94.61243348273837 178,3.841992200260431,0.9353244338700432,78.59571483731985,94.62443275717142 179,3.8163234885309785,0.9324265027864391,78.68970713238937,94.68842724698723 180,3.8143192613628547,0.9294543408191773,78.73970324134324,94.73642334870534 181,3.7683921229671427,0.9261675824185255,78.8036984342278,94.78241980570753 182,3.7818730481913394,0.923982460521124,78.83169595028589,94.84441482657462 183,3.755684365688915,0.9212717203776957,78.88969158285208,94.85041461209471 184,3.811672761406697,0.9191701354698967,78.96368532155611,94.87241293035424 185,3.7187630223556303,0.9165008361030031,79.07167678843993,94.91240963296178 186,3.7433232388026276,0.9128048724913043,79.16366926302634,94.94440711222595 187,3.783568885964407,0.9098437855447944,79.193666969411,94.97040483709546 188,3.7836115662480743,0.9066717340873534,79.25566220632525,94.9604057346799 189,3.7132954631053225,0.9037563167718037,79.32965616840008,94.99040315910469 190,3.7004594366315384,0.9007076963349615,79.47364497156146,95.0224008727251 191,3.7304701771534665,0.8978327910105387,79.55763772479744,95.08239607310907 192,3.700018348828168,0.8942069873439056,79.58763541287303,95.15439033187891 193,3.6864709585485325,0.8912874848430095,79.66762916962249,95.17638853296033 194,3.716994675112442,0.8881717415373188,79.77762017502955,95.21238559460166 195,3.6630856252052415,0.8846277142989999,79.83161617761498,95.24038335600144 196,3.70445132591355,0.8816261793859348,79.90161046942904,95.2903792196136 197,3.667574173967603,0.8784933531916568,79.9416069962694,95.34837442374727 198,3.673860294718138,0.8758927606294731,80.07559646265896,95.36637298386246 199,3.647094535156035,0.8726708035918772,80.16358933655913,95.41636914041993 200,3.6449320484215106,0.8685720142827263,80.24158311536738,95.41236961663184 201,3.6660440975511577,0.8651743567416157,80.33757523092001,95.43236791667019 202,3.6045285580863413,0.8614645302081125,80.4675654142859,95.4943630547154 203,3.6495057394806767,0.8582117335241934,80.51956098120304,95.52636053397954 204,3.6250970833738085,0.854794814142072,80.60955384036806,95.57435679315577 205,3.6242427893087896,0.8516537740072072,80.68154798195978,95.60035451802527 206,3.6249839084249147,0.8486535416422972,80.76154147505846,95.64635107389651 207,3.6208607680361036,0.8452714143509998,80.81553724328762,95.69034761154654 208,3.621414886394017,0.8423744141697912,80.89553079497537,95.71434561480199 209,3.595421079178931,0.8396461595297718,80.90353028196954,95.77434071631691 210,3.600605377009217,0.8370815479186103,81.0335199133166,95.81833705989068 211,3.6281545095040766,0.83375701421967,81.12551273943738,95.85633405919323 212,3.604720505190567,0.8295639896667458,81.24350308536596,95.89833075805107 213,3.5827912982081025,0.8255833837377787,81.4014905623346,95.91232946115288 214,3.567787086459952,0.8213720157159613,81.44348708542526,95.95632580472663 215,3.5835306644439697,0.817864267208721,81.5234805831012,96.00632172692787 216,3.5531847208318577,0.8139958393382087,81.59347526215237,96.02032093902214 217,3.556191263064532,0.8101979788756639,81.68346796111291,96.10231420500031 218,3.5177630505091706,0.8064951221806156,81.75546192693744,96.16030967461568 219,3.505100737155323,0.8028557199295516,81.83345549336033,96.17230883736332 220,3.548047834718731,0.7992166765078899,81.90345040035957,96.20830599787368 221,3.5126069230093084,0.7949462399827166,82.04543895002804,96.22230474125547 222,3.4366250877649014,0.791103566382353,82.14943049398578,96.26030164168898 223,3.5024204757851614,0.7875747270998349,82.24342289890978,96.31829690441171 224,3.52920873735992,0.7836198607651427,82.2774204330133,96.35229432133711 225,3.5523327135703933,0.7802904253530079,82.40141039784932,96.3882917345128 226,3.514156509453142,0.7771999123058131,82.44140739340219,96.42828861288753 227,3.480050802230835,0.7738232632063263,82.5373997726056,96.46428532299447 228,3.490331743804502,0.769954922446994,82.67138911083158,96.4842838207709 229,3.4522565519306023,0.7658971668519875,82.7413832342021,96.54027943877772 230,3.4249578193879464,0.7622650279282437,82.76738133532325,96.57427603545622 231,3.437175576115998,0.7588821934200403,82.83737553925373,96.60227404952136 232,3.4281330276542987,0.7550330743933857,82.89737097399396,96.63827128692986 233,3.412930045329349,0.7511200292064099,82.97936447432838,96.69026726397044 234,3.42685407316181,0.7473738250727845,83.14735111230164,96.70226630404724 235,3.4242665700509516,0.7433231376417255,83.16534943806057,96.74026295181542 236,3.406816643728337,0.7396762944652446,83.29333929652816,96.76226162160934 237,3.379652134129699,0.7358025643989741,83.37533269799354,96.80825790284433 238,3.421702680453448,0.7321584496476365,83.42132862769417,96.87225303713984 239,3.4100404222246627,0.7282632348175269,83.57931626212091,96.92824830361228 240,3.3646818852760423,0.7243552193070648,83.67330864507402,96.9402472851 241,3.376362484945378,0.7204217895032234,83.767300910849,96.99224302778434 242,3.3655178748386008,0.7166998286827423,83.88929142504537,97.04423923918118 243,3.366966569927377,0.7125734683561589,84.01128154742737,97.06023780304606 244,3.359389214448526,0.7084448325539596,84.1212729226779,97.10023482972436 245,3.358978704667427,0.7042798031884073,84.22926440054718,97.12023303455548 246,3.30417195172377,0.6996296609595207,84.33325582732678,97.1522305724087 247,3.296816946754993,0.6952631592550294,84.42524891709834,97.20822616295189

opened by GuideWsp 9
Question about apex

Hello ! Thanks for releasing the code ! While "--apex-amp" is included in the training command, "apex" is not mentioned in requirements. So I wonder whether the "apex amp" is used in your experiments. Besides, I would be grateful if you provide the training log in 8 V100 for me to check the correctness of my reproduction.

opened by ziyuwwang 3