Making available a high performing open-source European foundation model for fine-tuning -

Deliverables

• An EU wide digital infrastructure providing a high performing European large language foundation model for industrial and public services exploitation;
• Development of a strong community with the establishment of a coordination framework for the different stakeholders around development and exploitation of such foundation model.

Future developments in AI are expected to be based on the further use or fine tuning of large foundation models. It is therefore important that AI companies have full access to European trustworthy foundation models that include all the official languages of the Member States as well as the most socially and economically relevant ones and ensures transparency. Easy and long-term availability of foundation models in the EU would ensure European sovereignty in this area. Building on different European initiatives regarding foundation models would foster the creation of an open community around this topic, strengthening capabilities and consolidating resources, ensuring long term commitment by European actors to the development of transparent foundation models, and facilitating exploitation and fine tuning by European SMEs. The initiative aims to propose and make available one open-source large language foundation model as an infrastructure designed to be largely used by public or private users, and in particular by European SMEs, for further fine tuning. With this objective, the initiative will support the scaling up of an existing European foundation model and will release it under an open licence to maximize the impact on European Industry. Differently from existing open-source foundation models, all relevant components of such model benefitting from this action will be made available, such as properly anonymized training datasets, algorithms, and model weights. To consider security concerns, the model and its components will be licensed to European actors by a European entity (to be selected), established in the EU and not controlled by a third country. Possible entities could be the AI-on-Demand platform, a possible future EDIC or upon proposal from the consortium. The consortium that will carry out this action will comprise relevant developers including start-ups or RTOs leading in the domain, or a combination of these.

• European large language foundation model: this action will co-fund activities involving a number of developers, to scale-up a well selected, open-source foundation model to be made available to EU industrial and public services users and to the AI developer community, ensuring coverage of all the official languages of the Member States as well as the most socially and economically relevant ones, thereby maximizing the impact within the bounds of the technology. This model will be eventually applicable to broader types of uses and applications.
• Attention should be paid to the performance, transparency and security of the deployed model, through appropriate testing procedures, in compliance with the future AI Act.
• The EuroHPC Joint Undertaking would provide access to their facilities for the retaining of the model.
• Once developed, the model should be deployed and made available through the AI-on-Demand platform and the Alliance for Language Technologies action and/or other.
• The action should coordinate and build on related actions under Horizon Europe and Digital Europe Programme such as TrustLLM, HPLT, AI-on-Demand and the Alliance for Language Technologies action. Through this coordination the action should foster a coordination framework for the different stakeholders (e.g., Member States and industry) around relevant European, national, and private initiatives developing foundation models, ensuring complementary and synergies with communities supported by the Alliance for Language Technologies action.

News flashes