The proximity blast function operated by firing phasers in a concentrated bolt of energy, rather than a continuous beam. The phaser blast would then detonate at a predetermined point in space, affecting any ship or object within the blast radius.
The proximity blast setting was first described in the writer's guide, The Star Trek Guide (third revision, page 9).  When phaser fire is set for proximity explosion it acts somewhat like a "depth charge". The original special effect was also used to represent a photon torpedo launch on many occasions (giving the episode "Balance of Terror" a doubtful reputation in retrospect, as the occasion on which Captain Kirk repeatedly orders phaser fire but every time he does so the Enterprise appears to launch photon torpedoes).
To correct this, when the episode "Balance of Terror" was digitally remastered by CBS Digital in 2006, a unique special effect was developed.