The Stirling Engine was named by Dr. Rolf. J. Meijer who at that time was a project manager with Philips of Holland.  Philips was struggling with creating a new name to call the 'Air' engine when there was no air inside the engine.  This is because in an Air engine, the air inside the engine is called the 'working gas'.  If you change the 'working gas' to a gas like Helium or Hydrogen, then it can no longer be called an 'Air' engine.  The name Stirling Engine was chosen in honor of the inventor of the regenerator (economizer) and the engine that demonstrated its use.

The Stirling Engine's most basic configuration consists of two pistons each in its own cylinder.  (Sometimes it is easier to envision these two cylinders as one long tube with the piston heads facing each other inside the tube (see the figure below)). Note that between these two pistons heads are the heater, cooler and regenerator.  The regenerator (usually a block of woven wire) is in the center of this tube and the heater is between the regenerator and one piston (in red) while the cooler is between the regenerator and the other piston (in Blue). The volume attached to the 'heater' is the 'expansion space' where the hot gas pushes against the 'expansion piston'.  The volume attached to the 'cooler' is the 'compression space'.  
The regenerator is where the excess heat of the gas is stored in the regenerator  matrix on the way to the compression space from the expansion space and then the heat is recovered on the way back from the compression space to the expansion space.
 

Graphic courtesy of Dr. Israel Urieli of Ohio University.

Stirling Engine operation can be explained in a somewhat non technical way that applies to many but not to all engines that may be called Stirling Engines. 

We begin with the heater space and cooler space at their appropriate temperatures. The working gas trapped between the two piston heads is pushed by the Compression Piston through the regenerator where it is heated by the energy stored in the regenerator to the maximum temperature present in the regenerator  into the Heater volume where the gas expands due to increased temperature into the Expansion Space.  This results in an increased pressure pushing on the Expansion Piston so that it moves away from the regenerator pushing on a mechanism which changes the linear movement of the piston to a rotary motion.  This continues until all the gas that will expand has been pushed into the heater area and expanded.  The mechanism continues to push the Compression Piston further toward the Regenerator pushing all the gas out of the Compression Space into the gas circuit (heater, cooler, regenerator).

Once the expansion piston moves to its extreme position, the mechanism, to which both pistons are connected (but 90 rotary degrees apart), now begins to move the Expansion piston back the other way pushing the hot gas back through the Heater and then to the Regenerator and finally into the Cooler where it begins to Cool and contract (the pressure starts to drop).  The Compression Piston is also moving away from the regenerator while the Expansion piston comes toward the regenerator moving the gas through the regenerator into the compression space without compressing the gas.

The linkage continues to move the pistons until the Compression Piston is at its extreme and the Expansion piston is all the way forward.  At this point the mechanical arrangement moves the pistons together but because of the way the piston moves up and down in the cylinder but the mechanism is moving in a circle, the Expansion Piston  does not move very far but the Compression Piston moves toward the regenerator actually compressing the gas and begining to push the gas through the regenerator. (That is why it is called the Compression Piston.) 

 This brings us again to the first line of this explanation to complete the cycle and begin again.
 

Stirling Engineering (More Technical Explanation)

First Approximation of the power of a Stirling Engine (kinematic or free piston)
Power = (Beale.number) x (pressure(mean)) x (Volume Exp) x (frequency)
Watts = 0.116 x Pascals(10E-6) x (Cm^3) x (Hz)

A Stirling "Air" Engine is a mechanical device which operates on a closed regenerative thermodynamic cycle with cyclic compression and expansion of the working fluid (air) at different temperature levels. The flow of the working fluid is controlled by changes in the volume of the hot and cold spaces, eliminating the need for valves.  The Stirling Engine process (cycle) is reversible, meaning that an input of heat energy (burning fuel, for example) will produce an output of mechanical energy, and an input of mechanical energy (electric motor, etc.) will produce an output of heat energy. In this manner, the Stirling Engine can be used as a heat pump in much the same way as traditional refrigeration units, only using something other than the environmentally harmful refrigerants.

The most basic engine consists of a set of pistons, heat exchangers, and a device called a 'regenerator'. The engine is filled with a working fluid (gas) which is commonly Air, but some more advanced engines may use Nitrogen, Helium or Hydrogen. The pistons are arranged such that they create both a change in volume of the working fluid and create a net flow of the fluid through the heat exchangers. In this manner, heat is absorbed from an external source in the 'hot' end, creating mechanical energy, and rejected in the 'cold' end to the environment.

In a Stirling engine, the working fluid is completely contained inside the engine at all times, meaning the cycle is closed, As opposed to a typical gasoline engine, which takes in 'fresh air' for each new cycle. This enables a Stirling Engine to operate cleanly and quietly as there are no combustion products coming into contact with any of the engine's working components and no release of high-pressure gasses.

An important feature in Stirling Engines is the regenerator. On the most basic level, a regenerator is a device that absorbs heat from the working fluid as it enters the 'hot' end, and re-heats the fluid as it enters the 'cold' end. This internal recycling of energy allows for much higher efficiencies, and better performance overall. The regenerator is such a critical component that most Stirling Engines cannot operate efficiently without one!

Stirling Engineering ( Deeper understanding)
This link is a much deeper look into the theory. Click Here for a look at the Detailed Theory of Operation.
This information is from Dr. Israel Urieli of Ohio University.  Caution: Contains calculus, partial differential equations and thus requires a knowledge of the calculus. Also contains source code modules for a second order simulator (In 'C').