Basic Theory
A high voltage power supply charges up a capacitor C1 (kinda like a battery).  When the capacitor reaches a high enough voltage, the spark gap fires.  The spark gap is like a switch.  It goes on when the voltage gets high, and turns off when the voltage gets low.  When the spark gap fires, the energy stored up in the capacitor dumps into a 1:100 step-up transformer.  The primary (L1) is about 10 turns of heavy wire.  The secondary (L2)  is about 1000 turns of thin wire.  10:1000 or 1:100.  Feed in 10,000 volts, get out 1,000,000 volts.  ZZZZZZZAAAAAAPPP.  It all happens at a rate of over 120 times per second, often generating multiple discharges in many directions

Advanced Theory
 It is actually a bit more complicated then the explanation given above.  Although the turns ratio is around 1:100, the transformer operation is based more on resonance then on turns ratio.  When the spark gap fires, the electric charge in C1 dumps into L1, and then back into C1, and then back into L1.  This is called resonance.  C1 and L1 make up what is called a resonator.  They are changing an electric field (C1 volts) into a magnetic field (L1 gauss), and back again, at a rate (frequency) determined by the value of C1 x L1.  The secondary (L2) picks up some energy from L1 each time L1 charges up.  The output terminal C2 gets an electrical charge from L2 each time L2 discharges.  The secondary and the output terminal resonate at the frequency determined by L2 x C2.  The magic happens when L1 x C1 = L2 x C2, or both resonators resonate at the same rate  (this is made to happen by adjusting the tap on L1).  When both resonators are at the same rate, the energy in L2 builds by a little bit from L1 on each cycle.  This is called resonant rise.  The output terminal voltage gets higher on each cycle, until the voltage gets too high to hold, and then  ZZZZZZAAAAAAPPPP.

This is analogous to a person on a swing.  Imagine the legs are a resonator, going back and forth at a certain rate.  The swing with the person is another resonator, swinging back and forth at a certain rate.  Initially, the swing is hardly moving.  The legs start going back and forth and the swing starts going.  If done right, the legs change position at the very peaks of swing motion.  They are resonating at the same rate as the swing is swinging.  As each peak of swing motion is reached, the leg motion adds a little bit to the next cycle so that the swing arc grows a little each time.  This can continue until the arc gets over 180 degrees (horizontal at each peak).  Then, the chains begin to slacken and things can get pretty unpredictable.  In the same way, the resonator C1xL1 acts like the legs, adding a little energy to resonator C2xL2 (swing + person), on each cycle, until the voltage in C2 gets so high that it just explodes as an electrical discharge in search of ground.   Bwwwaaaahahahahahaha.

Unfortunately, even this is a bit of a simplification.  But that is as far as I am willing to go on theory.
Let's get on to the ZAP part.

Here is even more theory for those that gotta have it.

These things have been around for over 100 years.