In line with a number of other countries, Norway has decided to base their ICT solutions in the public sector on a common ICT architecture. This article discusses some challenges related to this work. The theoretical basis for the discussions is our understanding of information infrastructures, which we claim offers a fruitful perspective to the building of ICT architectures. Of particular relevance is its installed base: the history of technical and non-technical components that determines its further development. We argue that an ICT architecture for the public sector should be seen as an important element of a government information infrastructure. However, it has to be adapted to other principles and fulfil a wider range of needs than traditional types of infrastructures, including the specific political, regulatory and organizational context that it targets.