In detail
The High Commission's document gives the size as (2X2): two inches by two inches, or 51×51 mm. The photograph is square, not rectangular.
One piece of misinformation is worth knowing about: many blogs and agency pages state the size as 45×35 mm. That figure is not invented, but it belongs to the reverse case: Indian nationals applying for Bangladeshi visas. A 45×35 mm photo is not square and will not meet the requirement for an Indian visa from Bangladesh.
When you go to a studio, ask specifically for “two inches by two inches, for an Indian visa”. Asking only for a “passport size” photo may get you the size commonly used in Bangladesh.
IVAC’s page gives the size of the photo uploaded with the online application as 2×2 (350×350 pixels), so the pixel dimensions are square too.