Hey @Meher_Chandan,
In Strapi v4, it is actually not possible to extend the admin panel using the extensions folder.
You can however, create a custom plugin and override the default Dynamic Zone component (using app.addFields() in the register function) so you could then customize the ComponentPicker component.
If you are interested in that I can prepare a small tutorial/repo for it.
All the best 