Various Wikipedians have put together graphics tutorials. Some of these are still works in progress. It is preferred to work with open formats (such as PNG or SVG) and free licenses (such as public domain or GFDL) when possible. There are many free software programs available for creating and editing images.
PNGCrush - Old tried and true program that reduces PNG sizes using multiple zlib compression trials on disk.
OptiPNG - Essentially a newer version of PNGCrush, it economizes the trial (reducing from 100s of short ones to a maximum of 7 longer ones) and does everything in memory. Usually produces identical results, and is much faster.
AdvanceCOMP - A group of programs designed to reduce ZIP compression in many contexts, including PNGs (using AdvPNG).
Jhead - A program for viewing, altering, or removing Exif data of JPEGs. Amongst other things, this allows for reducing file sizes.
FileOptimizer - A WindowsGUI program (recommended for beginners) that uses multiple open source and freeware tools (including most of those listed here) to optimize a large number of media filetypes. Achieves better compression than any of the others used by themselves. Released under the LGPL.
PNGOUT - A program that optimizes PNGs by escewing the usual zlib compression for a slower but more complete algorithm.
PNG Gauntlet - GUI for OptiPNG, PNGOUT, and DeflOpt (similar to AdvPNG in AdvanceCOMP)
Image Analyzer - Amongst other things, allows for lossy compression of PNG files. Often, however, merely posterizing the image at a level that does not produce color banding results in smaller files of the same or higher quality.
JPG Cleaner - Removes Exif data from JPEGs, making the files smaller.
JPGExtra and PNGExtra - Programs that losslessly optimize JPEGs and PNGs by cleanly removing all non-image data.