PointCLIP:3D用CLIP预训练编码
Abstract
将CLIP学习到的2D表征迁移到3D领域来,在CLIP编码的点云和3D类别文本之间进行对齐
Method
![avatar](https://blog-img-1259433191.cos.ap-shanghai.myqcloud.com/PointCLIP/fig2.png)
关键就是找一个桥梁把3D和2D连接起来就行,把3D点云做了投射到2D平面上变成深度图,这个图像丢给CLIP视觉编码器得到表征
![avatar](https://blog-img-1259433191.cos.ap-shanghai.myqcloud.com/PointCLIP/fig3.png)
迁移到3D领域时融合的领域知识的trick
Experiments
![avatar](https://blog-img-1259433191.cos.ap-shanghai.myqcloud.com/PointCLIP/tab1-tab2.png)
![avatar](https://blog-img-1259433191.cos.ap-shanghai.myqcloud.com/PointCLIP/tab3-tab4.png)
![avatar](https://blog-img-1259433191.cos.ap-shanghai.myqcloud.com/PointCLIP/fig5.png)
![avatar](https://blog-img-1259433191.cos.ap-shanghai.myqcloud.com/PointCLIP/tab5.png)
![avatar](https://blog-img-1259433191.cos.ap-shanghai.myqcloud.com/PointCLIP/tab6-tab7.png)
将CLIP学习到的2D表征迁移到3D领域来,在CLIP编码的点云和3D类别文本之间进行对齐
关键就是找一个桥梁把3D和2D连接起来就行,把3D点云做了投射到2D平面上变成深度图,这个图像丢给CLIP视觉编码器得到表征
迁移到3D领域时融合的领域知识的trick