The World Bank's WDI database provides direct access to more than 900 development indicators, with time series for 210 countries from 1960 to present (where data are available). For more publicly available data, check out the World Banks open-data initiative.
The international organizations and sources listed here provide comprehensive statistics on many topics and for many countries (look for publications, interactive tools and downloadable files). When looking for data on a specific country, be sure to also check what official statistical agencies exist within the government.