[feat] Add UNITER pretraining heads#1126
[feat] Add UNITER pretraining heads#1126Ryan-Qiyu-Jiang wants to merge 11 commits intogh/ryan-qiyu-jiang/14/basefrom
Conversation
Add MRC, MRFR, WRA heads for UNITER pretraining. [ghstack-poisoned]
facebook-github-bot
added
the
CLA Signed
|
@Ryan-Qiyu-Jiang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Add MRC, MRFR, WRA heads for UNITER pretraining. Differential Revision: [D31768455](https://our.internmc.facebook.com/intern/diff/D31768455) [ghstack-poisoned]
|
@Ryan-Qiyu-Jiang has updated the pull request. You must reimport the pull request before landing. |
Add MRC, MRFR, WRA heads for UNITER pretraining. Differential Revision: [D31768455](https://our.internmc.facebook.com/intern/diff/D31768455) [ghstack-poisoned]
|
@Ryan-Qiyu-Jiang has updated the pull request. You must reimport the pull request before landing. |
|
@Ryan-Qiyu-Jiang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Add MRC, MRFR, WRA heads for UNITER pretraining. Differential Revision: [D31768455](https://our.internmc.facebook.com/intern/diff/D31768455) [ghstack-poisoned]
|
@Ryan-Qiyu-Jiang has updated the pull request. You must reimport the pull request before landing. |
Add MRC, MRFR, WRA heads for UNITER pretraining. Differential Revision: [D31768455](https://our.internmc.facebook.com/intern/diff/D31768455) [ghstack-poisoned]
|
@Ryan-Qiyu-Jiang has updated the pull request. You must reimport the pull request before landing. |
|
@Ryan-Qiyu-Jiang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Add MRC, MRFR, WRA heads for UNITER pretraining. Differential Revision: [D31768455](https://our.internmc.facebook.com/intern/diff/D31768455) [ghstack-poisoned]
|
@Ryan-Qiyu-Jiang has updated the pull request. You must reimport the pull request before landing. |
|
@Ryan-Qiyu-Jiang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Add MRC, MRFR, WRA heads for UNITER pretraining. Differential Revision: [D31768455](https://our.internmc.facebook.com/intern/diff/D31768455) [ghstack-poisoned]
|
@Ryan-Qiyu-Jiang has updated the pull request. You must reimport the pull request before landing. |
|
@Ryan-Qiyu-Jiang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Add MRC, MRFR, WRA heads for UNITER pretraining. Differential Revision: [D31768455](https://our.internmc.facebook.com/intern/diff/D31768455) [ghstack-poisoned]
|
@Ryan-Qiyu-Jiang has updated the pull request. You must reimport the pull request before landing. |
|
@Ryan-Qiyu-Jiang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Add MRC, MRFR, WRA heads for UNITER pretraining. Differential Revision: [D31768455](https://our.internmc.facebook.com/intern/diff/D31768455) [ghstack-poisoned]
|
@Ryan-Qiyu-Jiang has updated the pull request. You must reimport the pull request before landing. |
|
@Ryan-Qiyu-Jiang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Add MRC, MRFR, WRA heads for UNITER pretraining. Differential Revision: [D31768455](https://our.internmc.facebook.com/intern/diff/D31768455) [ghstack-poisoned]
|
@Ryan-Qiyu-Jiang has updated the pull request. You must reimport the pull request before landing. |
|
@Ryan-Qiyu-Jiang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Add MRC, MRFR, WRA heads for UNITER pretraining. Differential Revision: [D31768455](https://our.internmc.facebook.com/intern/diff/D31768455) [ghstack-poisoned]
|
@Ryan-Qiyu-Jiang has updated the pull request. You must reimport the pull request before landing. |
|
@Ryan-Qiyu-Jiang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Summary: Pull Request resolved: #1126 Add MRC, MRFR, WRA heads for UNITER pretraining. MRC = Masked Region Classification. MRFR = Masked Region Feature Regression. WRA = Word Region Alignment. Heads forward return a dict with `losses`. These heads can be used as pretraining tasks for other VL models. Details at https://arxiv.org/abs/1909.11740 Test Plan: **Unit tests** Test direct instantiation and forward pass for each head. Instantiation through build() and configs is tested in unit tests in later diffs by the models that use these heads. Tested as part of UNITER pretraining on masked COCO Reviewed By: ebsmothers Differential Revision: D31768455 Pulled By: Ryan-Qiyu-Jiang fbshipit-source-id: 9b48f81c472cd1859f32bc813484296208e206f5
2 participants
Stack from ghstack:
Add MRC, MRFR, WRA heads for UNITER pretraining.
Differential Revision: D31768455