In the KOSMOS-2 project, a smart learning approach in form of a modern portal for students at Universities has been developed, called “MeinKosmos”. An important question from the point of view of universities and teachers is what the quality of such new tools are or what their benefits are. This article describes the evaluation approach defined in the KOSMOS-2 project for the “MeinKosmos” learning portal, and the results of using the approach in an undergraduate degree program. The focus of the evaluation is on the learning success of the students and the quality of the meta-search.