Your garden hose spray nozzle might be the best tool for cleaning dirt off untreated, exterior brick. You can use a pressure washer, but stick with a lower-pressure nozzle since excessive force can damage masonry.
If deeper cleaning is needed, rinse the surface first, then scrub on a solution of water and trisodium phosphate (TSP) with a brush, following label directions carefully. Rinse again. TSP is sold in hardware stores and home centers.
Moss and algae can form on north-facing or heavily shaded brick. Both can be removed without damaging masonry. Rinse the surface, brush in a 1:1 solution of water and chlorine bleach and wait a while for the bleach to do its job. Rinse again and follow up with the cleaning procedures described above.
Wear skin and eye protection — a long-sleeved shirt, long pants and goggles — while working with either of these chemicals.
Reference:
Jeff Bredenberg, editor, in Clean It Fast, Clean It Right: The Ultimate Guide to Making Everything You Own Sparkle & Shine.