The main metrics would be:
Performance. This further breaks down to small-file performance (where latency, i.e. time to first byte dominates) and large-file performance (where transfer rate dominates). Obviously, actually measuring this is darn hard, but here are some decent benchmarks to get you started.
Geographical coverage for the regions you're interested in. Fx Akamai has > 65.000 servers worldwide and covers pretty much all continents except Antarctica; whereas MaxCDN doesn't have servers in Asia (yet).
Does the provider offer "Push" CDN zones, "Pull" zones, or both. Which do you want?
Does the provider offer streaming media zones (video/audio). Do you need this?
SSL. Not all providers support SSL / HTTPS. Of those who do, not all support SSL together with 'vanity domains' (
HTTP compression. Good CDNs allow for flexible control over HTTP compression (for much faster download of CSS, JS, and other text files). Amazon Cloudfront is the only exception I can think of right now.
Reporting. Good providers have great statistics, which help you get visibility into how your content is consumed. Smaller providers often have less extensive stats & reporting.
SLA's, for those who need this.
Does the provider offer Edge Side Includes, if you need this.
Does the provider offer some sort of token-based authentication (f.x. if you want to provide fast download of a software installer file, but won't allow download before the end user has paid for it).
You can find example of important CDN metrics in this comparation list. It includes information on pricing, additional payments (such as pricing for http request), how many PoPs (network location) can CDN offer you, what type of technology and features (https, video stream, live stream support, origin push) CDN supports and if it provides API. Which imho are important in CDN comparation. Linked list compared Cloudfront, MaxCDN, CDN77, Clouflare, Faslty and Cachefly. On the market is plenty of CDNs, but those mentioned are quitly nice picked to have a picture about what can be provided by CDN and to easily choose from them.
Useful help in decision process could be also web CDN Finder