Minutes of the TED eSender workshop (26 September 2024) 

These consolidated questions and answers have been reviewed and may have been regrouped or modified to provide a more cohesive and complete response. Some answers might no longer be accurate, as the Publications Office (OP) has updated its approach based on feedback.  

Recordings of the live sessions and presentation slides are available in the agenda section. 

Questions from the participants: 

• Can you confirm that SDK 1.13 is the first and only one compliant with the latest regulation from December 2023, or is SDK 1.12 also compliant? 

Answer: SDK 1.13 will be fully compliant with the latest eForms regulation amendment, including additions like IPI, Energy Efficiency Directive, and Review section. You’ll also need this version if you want to use the 6 new voluntary forms E1 to E6. While the regulation allows notices using SDK 1.13 from 1 November, SDK 1.12 remains compliant for exclusion and selection criteria if the new features aren’t required. We aim to release SDK 1.13 by the end of October.

[Update: release candidate 2 of SDK 1.13 is available at https://github.com/OP-TED/eForms-SDK/tree/1.13.0-rc.2 - the final version is planned for mid-November].

• How come the deadline for SDK 1.12 has been moved to July 2025? Didn't OP say earlier that by February 2025, a full switch to 1.13 was needed?

Answer: The deadline for SDK 1.12 has been extended to July 2025, as it remains compliant with the latest eForms regulation, particularly for exclusion and selection criteria. SDK 1.13 includes additional fields like IPI and Energy Efficiency, but if you’re not using these, SDK 1.12 will be accepted. While an initial switch to SDK 1.13 by February 2025 was suggested, this extension provides more flexibility for those not needing the newer features in SDK 1.13.

• Will Business Entities be mandatory in a future SDK version? Will elements that overlap with the purpose of Business Entities be removed?  

Answer: Business entities will become standard in SDK 2. They represent a better way to model relationships and handle cross-references in the system compared to the current model in SDK 1, which was deemed insufficient. Once SDK 2 is released, the older model will be phased out, and users will have time to migrate. 

SDK 1 will likely stay active for a while after SDK 2’s release, giving eSenders time to make the switch. The transition itself isn't overly complex, with the administrative side often being more of a challenge than the technical one. Your migration to SDK 2 is when you'll need to adopt business entities. 

• As a national authority, we have to provide continuous data through various bonds, which led us to build data-extracting services based on the current SDK's IDs and references. The switch to Business Entities is causing additional work. How can we handle this?

Answer: If we were to remove the references as they are currently used with business entities, you would need to adjust. However, you have ample time to do so until SDK 2. 

Regarding the changes to IDs in SDK 2, your reporting may need to adapt to using different methods for identifying objects. With SDK 2, we will be utilising business entities and removing certain fields. We understand that this may require some effort, but it is part of the evolution process. We strive to make changes in an organised and structured manner, which is why we have major versions. This allows us to ensure a smooth transition for our users. 

• The EC promised a longer lifespan for future SDK versions, but SDK 1.13 is set to last only until October 2025. Why?

Answer: In general, we design each SDK version to last about a year. However, we anticipate that SDK 1.13 might have a longer life – potentially extending into 2026, especially if we release SDK 2. For now, we've set the expiration date to October 2025 to give developers enough time to plan for future upgrades, but this may be extended depending on regulatory changes, feedback and user requirements.  

• Could technically optional additions in SDK 1.12 and 1.13 be required by some member state regulations for certain sectors? 

Answer: It depends on the specific country's regulations. For the FSR and IPI regulations, there's no national transposition, so your buyers could require them immediately. However, these types of notices are rare, and whether they affect you can also depend on the size of contracts in your country. The Energy Efficiency directive will likely take longer to come into effect, as it's subject to national transposition.

• In the Extended Lifespan, is maintenance support available for the SDK version if issues arise? 

Answer: During the first year of a version's lifespan, we provide patches and updates. Beyond that, we don't actively patch older SDK versions, but we will investigate and provide fixes for critical issues. Note that we won't make rule changes after the first year – if you need different rules, you will need to switch to a newer version. 

• We've been told that one day the TED eSender organisation indication in the notice content will be enforced. We currently allow customers to add it, but we'd like to automate it when it's enforced. In which SDK will this happen? 

Answer: The TED eSender organisation field will become mandatory in the future, but it's not included in SDK 1.13. We're still working on the details and haven't set a specific SDK for its implementation yet. However, adding it now is a good practice, as it makes it easier to identify the eSender in the XML and track on TED. Note that the email address and phone number are no longer mandatory in organisations (except buyers) since SDK 1.11, so you can skip these details if you wish for the “ted-esen” organisation.

• When will we have more details about the onsite summit, including the agenda and how to attend in person? 

Answer: The agenda is now available at https://op.europa.eu/en/web/ted-together. As for attending in person, we're providing a registration form, and once you're registered, we'll ensure access to the building. Please note that space is limited, so registration will close early. Also, please keep in mind that we won't be able to cover travel expenses, but we'll provide coffee and sandwiches during the event, which will take place at the Publications Office’s premises close to Luxembourg train station. The city is easily navigable with free public transport, which should help make your trip smoother. If you have any specific questions, feel free to reach out.

• When and where will the presentations from this webinar be available? 

Answer: The presentations and the link to the recording are always made available online as soon as possible on the respective “Agenda” tab of the event. See https://op.europa.eu/en/web/ted-eforms/agenda-q3-26-september-2024. 

• Given the ongoing SDK updates, can we expect API V2 to be supported until the end of 2025?

Answer: We plan to continue supporting API V2 with no immediate plans to remove it. While we recommend switching to API V3 at your earliest convenience, we won't force the transition unless absolutely necessary. We commit to maintaining API V2 support for at least the next six months, and it's likely to remain available for an extended period. Although some services may eventually become obsolete, we have no intention of cutting off API V2 support at this time. Feel free to plan your switch to API V3 at your convenience, and we'll ensure a seamless transition.

• When will the test environment be available with the new URLs? 

Answer: The test environment with API V3 is planned in Preview in November. We will provide an official announcement with the details.

• What are you doing to improve the performance and speed of the API, especially for long result notices? Contract award notices rendering API takes too much time and we get error (timeout) when notices have many lots (50 lots and more); can we have a solution or an alternative way to obtain the text, such as email?

Answer: We are aware of issues related to the API's performance and speed for long result notices. If you encounter these limitations, we recommend splitting large contract award notices into multiple submissions. For other types of notices, should you experience the 3-minute hard timeout limit during publication, please reach out to us so that we can investigate on a case-by-case basis. Additionally, you can refer to our FAQ, where we outline the factors that can impact submission times. 

In the next version of the viewer API, we will be implementing a feature to address rendering timeouts. If a visualisation takes longer than expected, we plan to introduce a notification system that alerts you when the process is complete. To further address slow rendering times for large notices, we are also introducing a caching mechanism in the TED API. This will ensure that once a notice is rendered and cached, subsequent API calls will return results instantly. However, during the current phases (EFX1 and SDK1), the first user to visualise a large notice may still experience some delay. 

We're continuously working on improving the overall API performance, especially for larger notices. This involves optimizing the XSLT transformations and refining the Schematron rules. Our collaboration with the Java libraries team has resulted in performance improvements for Schematron, and we're also enhancing the performance on the CVS side. Notice validation has become significantly faster in recent SDK versions, particularly in versions 1.10 and 1.13, so we strongly recommend upgrading to the latest SDK if you intend to publish very large notices. 

Looking ahead, we are planning the release of EFX2 and SDK2, which should also bring significant performance improvements, particularly for XSLT transformations. While this implementation is complex, we remain committed to ensuring the elimination of timeouts during notice processing. 

• If we send a notice using SDK 1.7 and get a "VALIDATION FAILED" error, is it acceptable to resend the notice using SDK 1.11 with an updated version ID?

Answer: Yes, that should work, though of course the SDK 1.11 rules would be applied to the next version of the notice. When you reuse the same notice ID and increase the version, the validation process simply re-validates the notice. The only element that's being reused is the ID itself, which shouldn't cause any issues. Additionally, each notice is validated individually, so you could theoretically even submit a completely different notice with the same ID as long as the version is different. 

As a reminder, you can also use the CVS API independently to validate your notices before submitting them for publication. Alternatively, you can run the Schematron validation locally to catch any errors before submission, which can help prevent validation errors at the submission point. 

• The rule that two separate organisations can't have the same national ID (BT-509) causes an issue: organisations with different addresses but with the same BT-509 can’t appear separately in the same notice. BT-509 in our country has a strict format so we can't change it. What should we do in this case? 

Answer: We assume that the question is about organisation ID BT-501. There is a rule that requires the organisation ID to be unique in each notice but it will be removed in SDK 1.13 and later backported to earlier active SDK versions. 

An alternative approach is to set up a main organisation with a national ID and create multiple touch points to represent different addresses. This way, you can establish a single organisational structure with different addresses for each touch point. However, please note that there may be restrictions on the roles or sub-roles that can be performed by touch points, so you'll need to consider these limitations.  

The reason we had this rule is to ensure that organisations are unique and can be easily identified using their registration numbers. We would still encourage buyers to avoid using multiple organisations if they are referring to the same one, which will reduce the effort to disambiguate and clean the data in later analysis and exploitation.  

• Will SDK 1.13 expire in October 2025? Will there be SDK 1.14 or 1.15 before February 2025?  

Answer: Currently, SDK 1.13 has been set to expire after 12 months but it will likely be extended. We haven’t decided the exact duration of the extension, which will depend on eSenders’ capacity to upgrade to this version. As for whether we'll have a version 1.14 or 1.15 before February or April 2025, we're currently focused on version 1.13 and haven't explored beyond that. However, we do have some ideas and features that we plan to carry over to version 1.14, and we'll likely build on those in the future. While it's unlikely that version 1.14 will be released by February, it's possible that it might be ready by April. 

We anticipate releasing a new SDK in the spring of next year, driven by maintenance needs and potential feedback from version 1.13. If we receive feedback from member states or encounter late-breaking issues that require changes, we'll need to implement those updates, which could potentially lead to a new SDK. By December, we should have a clearer understanding of our plans for next year, but for now, we're optimistic that the pace of change will be more manageable next year compared to this year. 

• We plan to upgrade from SDK 1.11 to SDK 1.12 in January 2025. Would it be better to upgrade directly to SDK 1.13? 

Answer: You can use 1.11 until the end of February 2025. However, it's not compliant with the latest amendment to the eForms regulation. If you're already working on 1.12 and given the limited effort between SDK 1.12 and 1.13, it would be advisable to upgrade directly to SDK 1.13. This will ensure you’re fully compliant with the latest regulations and avoid another upgrade shortly after. From an SDK point of view, this would be a more compliant solution.

• We are using SDK 1.10. Should we wait for SDK 1.13 or start development with SDK 1.12?

Answer: You can start with SDK 1.12 and prepare for the transition to SDK 1.13. Since the mandatory parts of SDK 1.12 won’t change, you can continue development and switch to SDK 1.13 when it’s available.  

• Would it be a good idea for the validation API in version 3 to return errors in JSON format?  

Answer: We have considered this idea, as it might allow us to provide more detailed and specific information in the validation report. But it requires careful design of the JSON structure, so that it’s useful and stable over the long term. We haven’t made any concrete progress on that topic yet. If you have specific ideas or suggestions on what a JSON validation report would look like, please post them on GitHub Discussions (https://github.com/OP-TED/eForms-SDK/discussions/categories/ideas).

• Do you have data on the average time it takes for eSenders to migrate between SDK versions? How has the metadata-driven approach affected the process? 

Answer: We don’t have hard data, but typically the first upgrade to an SDK is often the most challenging, with subsequent upgrades usually going more smoothly. The extent to which a system is metadata-driven plays a significant role in this process. Fully metadata-driven systems can automatically adjust to new SDKs, while those that are only partially metadata-driven may require more manual adjustments, such as mapping codelists. 

During the workshop, we conducted a live poll to gauge how long it takes eSenders to upgrade SDK versions once they begin the project. The results showed that 15% of participants complete upgrades in less than a month, 25% take 1-3 months, 45% take 4-6 months, and 15% take 6-12 months, with no one reporting timelines longer than 12 months. This suggests that most eSenders finish upgrades within 4 to 6 months of starting their upgrade projects. 

In June, we also collected information during workshop registration about migration timelines. The fastest migrations reported took a couple of months, even for those using metadata-driven solutions, which still require adjustments, deployment checks, and customisations. Six months was the most common time frame, especially from a technical perspective. Some eSenders reported migrations taking up to twelve months, often due to business considerations, but those longer timelines are rare and typically depend on factors like budget and project timing. Overall, metadata-driven migrations tend to be quicker. 

• Does SDK 1.12/1.13 allow a single subject to represent two different roles in the same form? For example, can a review body also be a buyer? Where can I find a link to the possible combination of roles?

Answer: While an organisation can hold different roles, certain combinations are prohibited. For example, a buyer cannot also be a tenderer, and as it stands, a buyer cannot act as a review body. If this is a possibility in your country, we should investigate it further. 

We also need to consider the identifiers for organisations and how touchpoints might interact with these identifiers. The same organisation can have one identifier but operate under different instances with various roles. There's a documented list of roles and sub-roles for touchpoints and organisations, which we'll check to ensure that the current application aligns with the written rules. If necessary, we can adjust a few constraints that may not be explicitly documented. 

• Our buyer sometimes also produces the products purchased, but they can’t currently act as both buyer and tenderer. Can this be addressed?

Answer: Technically you could get around the constraint by having two separate organisations in the notice. But it seems legally strange for a buyer to procure from themselves. We will consult with the European Commission if there are any cases where this could be allowed.

• Do you have plans to make a tool that converts JSON visual model to XML physical model? It is the most technically complex part of our eForms implementation, and it would be nice to use a standard tool for it. 

Answer: No, there is no such plan. Although the motivation for the question is understandable, generating a valid notice that adheres to the physical model from data that was collected through a user interface that adheres to the visual model (notice type definitions) is not a task that can be performed by a tool. This operation is at the core of every eSender application and there is no generic way that we can solve it once and for everyone. For example, in what programming language would we implement such algorithm? What runtime environment would this “tool” target? What assumptions would this tool make about the individual needs for tailoring eForms in each country? The role of the visual and physical models is to allow eSenders to implement their own XML notice generation in a way that operates across versions of the SDK. Additionally, we have provided details on the algorithm and logic of XML generation in the developer guide as well as a demo implementation in the Notice Editor sample application in GitHub. 

• Can we create a change notice from scratch (not from a published one)? If not, is there a way to add it next year please?  

Answer: From an API perspective, we don’t have strict validation rules yet in place. Technically, you could modify almost everything, but core fields—previously documented—should remain unchanged, and future rules will enforce this. 

Typically, a change notice modifies specific fields with a description, but you could allow users to start over, though this carries risks. Changes, such as altering lots, could violate directives. While the validation doesn’t block these changes now, allowing your users to make fundamental alterations is risky. Even if future rules limit field changes, other fields will still be modifiable. If you have a case where flexibility is needed, we can discuss, but for now, the system won't block these change notices, even if we don’t recommend it. 

-

Last update: 30 October 2024