One of the biggest challenges in the Telecommunications DevOps environments is the Validation and Verification (V&V) of Virtual Network Functions (VNF) and Network Services (NS) against different execution platforms. The V&V process allows service operators to have some level of confidence in third-party hosted code so they can be sure that VNFs and NSs will behave as expected immediately after they are deployed and put into production.

Similarly to the DevOps process, V&V is well-established in the software industry, where verification relates to compliance of the System Under Test (SUT) to the requirements (functional or not), whereas validation relates to testing whether it can address a (business) need.

As the issue of validation is critical, ETSI NFV ISG has released a number of documents approaching the need and the framework for pre-deployment validation and testing, as well as interoperability and portability. The issue of verification of NFV NSs is also addressed by NFV Research Group. The current outcomes of the working group mainly focus on the problem statement and the challenging issues that the verification of NSs faces. In addition to the above, standardisation/specification groups and open source initiatives are also addressing verification and validation, such as, for example, OPNFV. In the industry sector, validation frameworks for NFV environment and VNFs have already been proposed, with some organisations offering a portfolio of products addressing this topic.

As noted 5GTANGO is effectively an elaboration of the current trend to include testing frameworks in the Devops pipeline focusing on the needs of the telecom sector.

One of the main innovations to be achieved by 5GTANGO will be the implementation of a V&V platform providing ready-to-use exhaustive testing methods for third-party developed services over replicated operational infrastructures and under multiple and diverse testing conditions to assure the readiness of carrier-grade level network services. Such V&V components are not present in existing NFV architectures.

5GTANGO will provide a holistic breakthrough solution for validation and verification. Examples of verification tests for NFV/NS in 5GTANGO are; the check of descriptors, library dependencies, the integrity of service package, incomplete configurations, some functionality and integration interfaces, etc. Validation tests cover the deployment and also the test of features such as the necessary scalability, reliability, etc.

The objective of the 5GTANGO V&V platform is to provide a modular and vendor-independent facility to automatically test and validate VNFs and/or NSs submitted to the 5GTANGO Catalogue. Together with the Service Development Kit (SDK), the Catalogue and the Service Platform, the 5GTANGO ecosystem aims at providing an NFV DevOps model to accelerate the development and deployment of the new NFV services. This way, 5GTANGO will also contribute to lower the barrier to entry into the Telecom market to developers by providing mechanisms for an initial and automated qualification of services that guarantees the minimum quality required by network operators. Network operators will then open the door to a wider variety of partners.

The 5GTANGO V&V platform provides testing environments and tools, executes tests and provides signed results. This also includes the sandbox environment available to the developers for testing during development time (not verification).

Such a V&V process does not only include functional testing of VNFs and NSs but also non-functional tests, such as performance measurements for gaining insights about resource requirements to fulfil SLAs and to provide the expected Quality of Experience (QoE). To fit seamlessly into the anticipated DevOps workflow, all these V&V procedures need to be fully automated and be able to qualify any VNF or NS without further human interaction.

The tests needed for V&V can be generated automatically using MBT (Model-Based Testing). Tests are categorized and stored in a repository to facilitate the composition of the test suites and the prioritization of the test execution if the resources are limited. Test results must provide as complete as possible information on the SUT and offer a direct comparability between different implementations of a same VNF.

In the proposed V&V platform, we decide to use the ETSI normalised TTCN-3, Testing and Test Control Notation version 3, language for test development, generation and execution. TTCN-3 is a test scripting language widely known in the telecommunication sector. It is used by the Third Generation Partnership Project (3GPP) for interoperability and certification testing, including the prestigious test suite for Long Term Evolution (LTE)/4G terminals. Using TTCN-3 will further increase the exploitation potential of 5GTANGO results.

Certifications are organized all over the world in mostly all industrial domains either for regulators or for organizations on a voluntary basis. The aim of a certification process is to ascertain the conformity which is defined as the fact that a product, system, body or even a person meets specified requirements and which can improve the business interests with regard to products, goods and services. The 5GTANGO V&V framework aims to deliver such a certificate to the VNF/NS under test if they meet all the requirements. 5GTANGO proposes a pre-defined set of automated tests as a certification process for complying network services of VNFs vendors and third-party developers.

The V&V is not intended as a debugging solution for developers. Debugging is a huge task worthy of detailed analysis in its own right and, therefore, beyond the scope of 5GTANGO. However, the V&V results can be used to support a developer in RCA (Root Cause Analysis) scenarios. It is expected that, when a submitted VNF/NS fails the verification and validation, a brief report is provided with the failure and the justification. It is envisioned that the developer may need to recreate a test scenario in order to better understand the root cause of the failure and resolve it. The V&V will support the replicability of tests by recording generated data and system configuration information and allowing this to be “played back". In this way, a developer can recreate a test in their own environment for debugging purposes.

Monitoring functionality is also essential to collect the VNF/NS performance data and to achieve state-of-the-art validation focused on VNF and NS.

The V&V platform is a single component of the 5GTANGO project and it is being designed to be generic and easily adaptable to all NFV-compliant infrastructures. This way, it will be easily replicated in any operator's NFV infrastructure or even multiple-operator infrastructure. To interact with a V&V platform, a simple interface is provided that can be used by any role (i.e. the developer) or component (i.e the catalogue) in a similar fashion.