You're skipping about 1000 years of history there to make your point. By the time of the rise of nationalism, many parts of Europe had shifted back and forth in control between larger empires for hundreds of years, and it mostly was not a continent of tiny little city states and small kingdoms any longer. The era of the city states started fading quickly after the renaissance, and were in the first place mostly limited to small parts of Europe - most of Europe had seen ongoing consolidation of smaller kingdoms for a millennia or two by the time nationalism became a factor.
Even of the smaller states, many were parts of larger empires for most of their existence, though the control might shift back and forth between empires.
E.g. Germany did not exist as a united country, sure, but the Holy Roman Empire (not to be confused with the Roman Empire) encompassed most of the smaller countries on the territories of present day Germany and at various times part of Italy, Belgium, Netherlands, Austria, France, Switzerland, Poland, Croatia, Slovenia, Czech Republic. Even within the Holy Roman Empire, many of the constituent states were at various times larger than their succeeding present day European states (though there were also many smaller states).
Nationalism in many instances erected substantial new borders in Europe in places where borders meant little to ordinary people before.
> You're skipping about 1000 years of history there to make your point.
This varied by area. Some nation states developed early (England, France, Spain, Portugal, Denmark, Sweden, Switzerland), while other areas was dominated by city states much longer, like Italy and Germany that did not become a nation states before around 1870.
For the most part these countries were monarchies whose territories waxed and waned with the power of their ruling houses, where the idea of a nation with a specific government did not develop much earlier than elsewhere. Their development into nation states in fact largely happened as they contracted into core territories with common cultures and languages and the power of their monarchs was put under increasing challenge.
As for Germany and Italy, while they did not become nation states until late in the game, a large part of that was that they were part of empires and other kingdoms long before.
The Holy Roman Empire (not to be confused with the Roman Empire), the West Roman Empire, the Ostrogothic Kingdom ("heir" to the West Roman Empire, including Italy), the Frankish (Carolingian) Empire, the Bysantine Empire (East Roman Empire); Habsburg Spain; Habsburg Austria; the original Roman Empire.
The period of scattered small autonomous city states was a historical "blip" of not much more than two hundred years due to a temporary political and military vacuum caused by the constant tug of war of various of these larger empires.
The core areas of Germany and Italy had been dominated by organized larger scale governments for the most part of the preceding two millenias (depending on region) before nationalism existed.
See this map from 1713, for example: http://victoriavane.files.wordpress.com/2011/08/western_euro...
Even of the smaller states, many were parts of larger empires for most of their existence, though the control might shift back and forth between empires.
E.g. Germany did not exist as a united country, sure, but the Holy Roman Empire (not to be confused with the Roman Empire) encompassed most of the smaller countries on the territories of present day Germany and at various times part of Italy, Belgium, Netherlands, Austria, France, Switzerland, Poland, Croatia, Slovenia, Czech Republic. Even within the Holy Roman Empire, many of the constituent states were at various times larger than their succeeding present day European states (though there were also many smaller states).
Nationalism in many instances erected substantial new borders in Europe in places where borders meant little to ordinary people before.