Mathematically, the term "dimension" here refers to a "type" adjoined to a real number. For example, 3.25 meters has the real number 3.25 and a "meters" type. The real part of the pair is a member of the field of real numbers, the type part has a group structure. Physical quantities must be described using both, and have the mathematical structure of a typed family of fields. The reason this distinction matters can be illustrated using an analogy: you can't add apples and oranges, but you can multiply or divide them. Thus the expression 4 meters + 3 seconds has no physical meaning, but (4/3)m/s is a perfectly reasonable velocity.
In common usage, the dimension of a physical quantity is the type of units needed to express it. For instance, the dimension of a speed is length/time, the dimension of a force is mass*length/time2. In mechanics, every dimension can be expressed in terms of length, time, and mass, or in terms of force, length and mass. These two form fundamentally different systems (note to self: where are Barenblatts notes?) related by F = ma. Each system has strengths and weaknesses, picking the right system can substantially reduce the effort necessary for solving a problem. For example, FLT lends itself well to [minimum potential energy]? techniques because of the simple form of units of energy in this system: FL.
In the most primitive form, dimensional analysis is used to check the correctness of algebraic derivations: in every physically meaningful equation, the dimensions of the two sides must be identical. Furthermore, the arguments to exponential, trigonometric and logarithmic functions must be dimensionless numbers, which is typically achieved by multiplying a certain physical quantity by a suitable constant of the inverse dimension.
Note also that the dimensionless numbers are not really dimensionless. The actual structure of a dimensionless number is unity in the type. For example, consider the so-called dimensionless unit of strain: L/L. The L/L units are usually dropped, either implicitly or explicitly, but it is a mistake to regard strain as a physically meaningful quantity without some notion of the L in the denominator, which acts as a gauge length. For another example, consider the physical meaning (none)of adding strain (dimensionless) to Mach (dimensionless).
The above mentioned reduction of variables uses the [Buckingham Pi theorem]? as its central tool. This theorem describes how an equation involving several variables can be equivalently rewritten as an equation of fewer dimensionless parameters, and it provides a method for computing these dimensionless parameters from the given variables, even if the form of the equation is still unknown. Two systems for which these parameters coincide are then equivalent for the purposes of the equation, and the experimentalist who wants to determine the form of the equation can choose the most convenient one.
A typical application of dimensional analysis occurs in fluid dynamics. If a moving fluid meets an object, it exerts a force on the object, according to a complicated (and not completely understood) law (transfer of momentum?). The variables involved are: the speed, density and viscosity of the fluid, the size and shape of the body, and the force. Using the algorithm of the pi theorem, one can defined two dimensionless parameters from these variables: the drag coefficient and the [Reynolds number]?. The original law is then reduced to a law involving only these two numbers. To empirically determine this law, instead of experimenting on huge bodies with fast flowing fluids, one may just as well experiment on small models with slow flowing, more viscous fluids, because these two systems are described by the same set of dimensionless parameters.