diff --git a/.gitbook/assets b/.gitbook/assets deleted file mode 120000 index e4c5bd02..00000000 --- a/.gitbook/assets +++ /dev/null @@ -1 +0,0 @@ -../images/ \ No newline at end of file diff --git a/images/gb-cover-final.png b/.gitbook/assets/gb-cover-final.png similarity index 100% rename from images/gb-cover-final.png rename to .gitbook/assets/gb-cover-final.png diff --git a/LICENSE b/LICENSE deleted file mode 100644 index 261eeb9e..00000000 --- a/LICENSE +++ /dev/null @@ -1,201 +0,0 @@ - Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS - - APPENDIX: How to apply the Apache License to your work. - - To apply the Apache License to your work, attach the following - boilerplate notice, with the fields enclosed by brackets "[]" - replaced with your own identifying information. (Don't include - the brackets!) The text should be enclosed in the appropriate - comment syntax for the file format. We also recommend that a - file or class name and description of purpose be included on the - same "printed page" as the copyright notice for easier - identification within third-party archives. - - Copyright [yyyy] [name of copyright owner] - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. diff --git a/README.md b/README.md index 984c1ca2..9a5f64a4 100644 --- a/README.md +++ b/README.md @@ -1,42 +1,40 @@ --- -description: Get started with the Cisco Crosswork NSO documentation guides. -icon: power-off -cover: images/gb-cover-final.png -coverY: -33.22891656662665 +description: Cisco-provided NED documentation. +icon: paper-plane +cover: .gitbook/assets/gb-cover-final.png +coverY: -34.02798619447779 --- -# Start +# Overview -Use this page to navigate your way through the NSO documentation and access the resources most relevant to your role. +**Cisco NSO (Network Services Orchestrator) NEDs (Network Element Drivers)** are software components that enable Cisco NSO to communicate with and configure network devices from different vendors using their native CLI, NETCONF, RESTCONF, or other proprietary interfaces.\ +NEDs translate the NSO service models into device-specific commands, allowing NSO to manage multi-vendor networks efficiently. -## NSO Roles +## NSO NED Administration -An NSO deployment typically consists of the following roles: +See the [NSO Administration Guide](https://cisco-tailf.gitbook.io/nso-docs/guides/administration/management/ned-administration) to learn about Cisco-provided NEDs and how to manage them. -
AdministratorsPersonnel who deploy & manage an NSO deployment.
OperatorsPersonnel who use & operate an NSO deployment.
DevelopersPersonnel who develop NSO services, packages, & more.
+## The NED `README.md` File -## Learn NSO +Each NED package comes with a `README.md` file that provides essential documentation and details, including: -For users new to NSO or wanting to explore it further. +1. **Overview of the NED** + * A brief introduction to the NED, including its supported device types and software versions. +2. **Installation Instructions** + * Steps for installing and configuring the NED in Cisco NSO. +3. **Supported Interfaces and Protocols** + * Specifies whether the NED supports CLI, NETCONF, RESTCONF, or other management protocols. +4. **Feature Support List** + * Lists supported commands, configurations, and features for the device. +5. **Limitations and Known Issues** + * Any constraints, unsupported features, or known bugs related to the NED. +6. **Usage Instructions** + * Example commands, data models, and guidelines on how to interact with the NED. +7. **Upgrade and Compatibility Information** + * Details on how to upgrade the NED and which Cisco NSO versions it is compatible with. +8. **Licensing and Support Information** + * Guidelines on licensing requirements and where to get support. -
NSO at a GlanceA 20,000-foot view of NSO components and concepts.https://nso-docs.cisco.com/nso-basics/nso-at-a-glance
Solution OverviewNSO overview & how it meets automation needs.https://www.cisco.com/c/en/us/products/collateral/cloud-systems-management/network-services-orchestrator/network-orchestrator-so.html
Learning LabsDeep dive into NSO with hands-on learning modules.https://developer.cisco.com/learning/search/?contentType=track,module,lab&keyword=nso&sortBy=luceneScore
+## Cisco NSO NED Changelog Explorer -{% hint style="info" %} -A more comprehensive list of learning resources and associated material is available on the [Learning Paths](https://nso-docs.cisco.com/learn-nso/learning-paths) page. -{% endhint %} - -## Work with NSO - -For users working in a production-wide NSO deployment. - -### Administration - -
Installation & DeploymentPlan, install, and upgrade your NSO deployment.#installation-and-deployment
ManagementAdministrate and manage your NSO deployment.#management
Advanced TopicsDelve into advanced NSO topics.#advanced-topics
- -### Operation and Usage - -
CLIGet started with the NSO CLI and base concepts.#cli
Web UIOperate & interact with NSO using the Web UI.#web-ui
OperationsPerform different NSO operations.#operations
- -### Development - -
Introduction to AutomationDevelop basic NSO automation understanding.#introduction-to-automation
Core ConceptsMain concepts in NSO development.#core-concepts
Advanced DevelopmentDeep dive into advanced development topics.#advanced-development
Connected TopicsTopics connected to NSO development.#connected-topics
+The [NED Changelog Explorer](https://developer.cisco.com/docs/nso/ned-changelog-explorer/) allows you to quickly search for changes when upgrading to a specific NED version. This information is also available in the CHANGES file, packaged with each NSO NED release. diff --git a/SUMMARY.md b/SUMMARY.md index 736ea8f5..066585cf 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -1,200 +1,369 @@ # Table of contents -* [Start](README.md) -* [What's New](whats-new.md) - -## Administration - -* [Get Started](administration/get-started.md) -* [Installation and Deployment](administration/installation-and-deployment/README.md) - * [Local Install](administration/installation-and-deployment/local-install.md) - * [System Install](administration/installation-and-deployment/system-install.md) - * [Post-Install Actions](administration/installation-and-deployment/post-install-actions/README.md) - * [Explore the Installation](administration/installation-and-deployment/post-install-actions/explore-the-installation.md) - * [Start and Stop NSO](administration/installation-and-deployment/post-install-actions/start-stop-nso.md) - * [Create NSO Instance](administration/installation-and-deployment/post-install-actions/create-nso-instance.md) - * [Enable Development Mode](administration/installation-and-deployment/post-install-actions/enable-development-mode.md) - * [Running NSO Examples](administration/installation-and-deployment/post-install-actions/running-nso-examples.md) - * [Migrate to System Install](administration/installation-and-deployment/post-install-actions/migrate-to-system-install.md) - * [Modify Examples for System Install](administration/installation-and-deployment/post-install-actions/modify-examples-for-system-install.md) - * [Uninstall Local Install](administration/installation-and-deployment/post-install-actions/uninstall-local-install.md) - * [Uninstall System Install](administration/installation-and-deployment/post-install-actions/uninstall-system-install.md) - * [Containerized NSO](administration/installation-and-deployment/containerized-nso.md) - * [Development to Production Deployment](administration/installation-and-deployment/development-to-production-deployment/README.md) - * [Develop and Deploy a Nano Service](administration/installation-and-deployment/deployment/develop-and-deploy-a-nano-service.md) - * [Secure Deployment](administration/installation-and-deployment/deployment/secure-deployment.md) - * [Deployment Example](administration/installation-and-deployment/deployment/deployment-example.md) - * [Upgrade NSO](administration/installation-and-deployment/upgrade-nso.md) -* [Management](administration/management/README.md) - * [System Management](administration/management/system-management/README.md) - * [Cisco Smart Licensing](administration/management/system-management/cisco-smart-licensing.md) - * [Log Messages and Formats](administration/management/system-management/log-messages-and-formats.md) - * [Alarm Types](administration/management/system-management/alarms.md) - * [Package Management](administration/management/package-mgmt.md) - * [High Availability](administration/management/high-availability.md) - * [AAA Infrastructure](administration/management/aaa-infrastructure.md) - * [NED Administration](administration/management/ned-administration.md) -* [Advanced Topics](administration/advanced-topics/README.md) - * [Locks](administration/advanced-topics/locks.md) - * [CDB Persistence](administration/advanced-topics/cdb-persistence.md) - * [IPC Connection](administration/advanced-topics/ipc-connection.md) - * [Cryptographic Keys](administration/advanced-topics/cryptographic-keys.md) - * [Service Manager Restart](administration/advanced-topics/restart-strategies-for-service-manager.md) - * [IPv6 on Northbound Interfaces](administration/advanced-topics/ipv6-on-northbound-interfaces.md) - * [Layered Service Architecture](administration/advanced-topics/layered-service-architecture.md) - -## Operation & Usage - -* [Get Started](operation-and-usage/get-started.md) -* [CLI](operation-and-usage/cli/README.md) - * [Introduction to NSO CLI](operation-and-usage/cli/introduction-to-nso-cli.md) - * [CLI Commands](operation-and-usage/cli/cli-commands.md) -* [Web UI](operation-and-usage/webui/README.md) - * [Home](operation-and-usage/webui/home.md) - * [Devices](operation-and-usage/webui/devices.md) - * [Services](operation-and-usage/webui/services.md) - * [Config Editor](operation-and-usage/webui/config-editor.md) - * [Tools](operation-and-usage/webui/tools.md) -* [Operations](operation-and-usage/operations/README.md) - * [Basic Operations](operation-and-usage/operations/basic-operations.md) - * [NEDs and Adding Devices](operation-and-usage/operations/neds-and-adding-devices.md) - * [Manage Network Services](operation-and-usage/operations/managing-network-services.md) - * [Device Manager](operation-and-usage/operations/nso-device-manager.md) - * [Out-of-band Interoperation](operation-and-usage/operations/out-of-band-interoperation.md) - * [SSH Key Management](operation-and-usage/operations/ssh-key-management.md) - * [Alarm Manager](operation-and-usage/operations/alarm-manager.md) - * [Plug-and-Play Scripting](operation-and-usage/operations/plug-and-play-scripting.md) - * [Compliance Reporting](operation-and-usage/operations/compliance-reporting.md) - * [Listing Packages](operation-and-usage/operations/listing-packages.md) - * [Lifecycle Operations](operation-and-usage/operations/lifecycle-operations.md) - * [Network Simulator](operation-and-usage/operations/network-simulator-netsim.md) - -## Development - -* [Get Started](development/get-started.md) -* [Introduction to Automation](development/introduction-to-automation/README.md) - * [CDB and YANG](development/introduction-to-automation/cdb-and-yang.md) - * [Basic Automation with Python](development/introduction-to-automation/basic-automation-with-python.md) - * [Develop a Simple Service](development/introduction-to-automation/develop-a-simple-service.md) - * [Applications in NSO](development/introduction-to-automation/applications-in-nso.md) -* [Core Concepts](development/core-concepts/README.md) - * [Services](development/core-concepts/services.md) - * [Implementing Services](development/core-concepts/implementing-services.md) - * [Templates](development/core-concepts/templates.md) - * [Nano Services](development/core-concepts/nano-services.md) - * [Packages](development/core-concepts/packages.md) - * [Using CDB](development/core-concepts/using-cdb.md) - * [YANG](development/core-concepts/yang.md) - * [NSO Concurrency Model](development/core-concepts/nso-concurrency-model.md) - * [Service Handling of Ambiguous Device Models](development/core-concepts/service-handling-of-ambiguous-device-models.md) - * [NSO Virtual Machines](development/core-concepts/nso-virtual-machines/README.md) - * [NSO Python VM](development/core-concepts/nso-virtual-machines/nso-python-vm.md) - * [NSO Java VM](development/core-concepts/nso-virtual-machines/nso-java-vm.md) - * [Embedded Erlang Applications](development/core-concepts/nso-virtual-machines/embedded-erlang-applications.md) - * [API Overview](development/core-concepts/api-overview/README.md) - * [Python API Overview](development/core-concepts/api-overview/python-api-overview.md) - * [Java API Overview](development/core-concepts/api-overview/java-api-overview.md) - * [Northbound APIs](development/core-concepts/northbound-apis/README.md) - * [NSO NETCONF Server](development/core-concepts/northbound-apis/nso-netconf-server.md) - * [RESTCONF API](development/core-concepts/northbound-apis/restconf-api.md) - * [NSO SNMP Agent](development/core-concepts/northbound-apis/nso-snmp-agent.md) -* [Advanced Development](development/advanced-development/README.md) - * [Development Environment and Resources](development/advanced-development/development-environment-and-resources.md) - * [Developing Services](development/advanced-development/developing-services/README.md) - * [Services Deep Dive](development/advanced-development/developing-services/services-deep-dive.md) - * [Service Development Using Java](development/advanced-development/developing-services/service-development-using-java.md) - * [NSO Developer Studio](https://nso-docs.cisco.com/resources/platform-tools/nso-developer-studio) - * [Developing Packages](development/advanced-development/developing-packages.md) - * [Developing NEDs](development/advanced-development/developing-neds/README.md) - * [NETCONF NED Development](development/advanced-development/developing-neds/netconf-ned-development.md) - * [CLI NED Development](development/advanced-development/developing-neds/cli-ned-development.md) - * [Generic NED Development](development/advanced-development/developing-neds/generic-ned-development.md) - * [SNMP NED](development/advanced-development/developing-neds/snmp-ned.md) - * [NED Upgrades and Migration](development/advanced-development/developing-neds/ned-upgrades-and-migration.md) - * [Developing Alarm Applications](development/advanced-development/developing-alarm-applications.md) - * [Kicker](development/advanced-development/kicker.md) - * [Scaling and Performance Optimization](development/advanced-development/scaling-and-performance-optimization.md) - * [Progress Trace](development/advanced-development/progress-trace.md) - * [Web UI Development](development/advanced-development/web-ui-development/README.md) - * [JSON-RPC API](development/advanced-development/web-ui-development/json-rpc-api.md) -* [Connected Topics](development/connected-topics/README.md) - * [SNMP Notification Receiver](development/connected-topics/snmp-notification-receiver.md) - * [Web Server](development/connected-topics/web-server.md) - * [Scheduler](development/connected-topics/scheduler.md) - * [External Logging](development/connected-topics/external-logging.md) - * [Encrypted Strings](development/connected-topics/encryption-keys.md) - -## Resources - -* [Manual Pages](resources/man/README.md) - * [clispec](resources/man/clispec.5.md) - * [confd\_lib](resources/man/confd_lib.3.md) - * [confd\_lib\_cdb](resources/man/confd_lib_cdb.3.md) - * [confd\_lib\_dp](resources/man/confd_lib_dp.3.md) - * [confd\_lib\_events](resources/man/confd_lib_events.3.md) - * [confd\_lib\_ha](resources/man/confd_lib_ha.3.md) - * [confd\_lib\_lib](resources/man/confd_lib_lib.3.md) - * [confd\_lib\_maapi](resources/man/confd_lib_maapi.3.md) - * [confd\_types](resources/man/confd_types.3.md) - * [mib\_annotations](resources/man/mib_annotations.5.md) - * [ncs](resources/man/ncs.1.md) - * [ncs-backup](resources/man/ncs-backup.1.md) - * [ncs-collect-tech-report](resources/man/ncs-collect-tech-report.1.md) - * [ncs-installer](resources/man/ncs-installer.1.md) - * [ncs-maapi](resources/man/ncs-maapi.1.md) - * [ncs-make-package](resources/man/ncs-make-package.1.md) - * [ncs-netsim](resources/man/ncs-netsim.1.md) - * [ncs-project](resources/man/ncs-project.1.md) - * [ncs-project-create](resources/man/ncs-project-create.1.md) - * [ncs-project-export](resources/man/ncs-project-export.1.md) - * [ncs-project-git](resources/man/ncs-project-git.1.md) - * [ncs-project-setup](resources/man/ncs-project-setup.1.md) - * [ncs-project-update](resources/man/ncs-project-update.1.md) - * [ncs-setup](resources/man/ncs-setup.1.md) - * [ncs-uninstall](resources/man/ncs-uninstall.1.md) - * [ncs.conf](resources/man/ncs.conf.5.md) - * [ncs\_cli](resources/man/ncs_cli.1.md) - * [ncs\_cmd](resources/man/ncs_cmd.1.md) - * [ncs\_load](resources/man/ncs_load.1.md) - * [ncsc](resources/man/ncsc.1.md) - * [tailf\_yang\_cli\_extensions](resources/man/tailf_yang_cli_extensions.5.md) - * [tailf\_yang\_extensions](resources/man/tailf_yang_extensions.5.md) - -## Developer Reference - -* [Python API Reference](developer-reference/pyapi/README.md) - * [ncs Module](developer-reference/pyapi/ncs.md) - * [ncs.alarm Module](developer-reference/pyapi/ncs.alarm.md) - * [ncs.application Module](developer-reference/pyapi/ncs.application.md) - * [ncs.cdb Module](developer-reference/pyapi/ncs.cdb.md) - * [ncs.dp Module](developer-reference/pyapi/ncs.dp.md) - * [ncs.experimental Module](developer-reference/pyapi/ncs.experimental.md) - * [ncs.log Module](developer-reference/pyapi/ncs.log.md) - * [ncs.maagic Module](developer-reference/pyapi/ncs.maagic.md) - * [ncs.maapi Module](developer-reference/pyapi/ncs.maapi.md) - * [ncs.progress Module](developer-reference/pyapi/ncs.progress.md) - * [ncs.service\_log Module](developer-reference/pyapi/ncs.service_log.md) - * [ncs.template Module](developer-reference/pyapi/ncs.template.md) - * [ncs.util Module](developer-reference/pyapi/ncs.util.md) - * [\_ncs Module](developer-reference/pyapi/_ncs.md) - * [\_ncs.cdb Module](developer-reference/pyapi/_ncs.cdb.md) - * [\_ncs.dp Module](developer-reference/pyapi/_ncs.dp.md) - * [\_ncs.error Module](developer-reference/pyapi/_ncs.error.md) - * [\_ncs.events Module](developer-reference/pyapi/_ncs.events.md) - * [\_ncs.ha Module](developer-reference/pyapi/_ncs.ha.md) - * [\_ncs.maapi Module](developer-reference/pyapi/_ncs.maapi.md) -* [Java API Reference](developer-reference/java-api-reference.md) -* [Erlang API Reference](developer-reference/erlang/README.md) - * [econfd Module](developer-reference/erlang/econfd.md) - * [econfd_cdb Module](developer-reference/erlang/econfd_cdb.md) - * [econfd_ha Module](developer-reference/erlang/econfd_ha.md) - * [econfd_logsyms Module](developer-reference/erlang/econfd_logsyms.md) - * [econfd_maapi Module](developer-reference/erlang/econfd_maapi.md) - * [econfd_notif Module](developer-reference/erlang/econfd_notif.md) - * [econfd_schema Module](developer-reference/erlang/econfd_schema.md) -* [RESTCONF API](developer-reference/restconf-api/README.md) - * [Sample RESTCONF API Docs](https://developer.cisco.com/docs/nso/overview/) -* [NETCONF Interface](developer-reference/netconf-interface.md) -* [JSON-RPC API](developer-reference/json-rpc-api.md) -* [SNMP Agent](developer-reference/snmp-agent.md) -* [XPath](developer-reference/xpath.md) +* [Overview](README.md) + +## Cisco-provided NEDs +* a10-acos + * [README-ned-settings](a10-acos/README-ned-settings.md) + * [README v3.26 2025-11-12](a10-acos/README.md) + +* accedian-nid + * [README-ned-settings](accedian-nid/README-ned-settings.md) + * [README v4.6.1 2025-07-03](accedian-nid/README.md) + +* accedian-skylight_rc + * [README-ned-settings](accedian-skylight_rc/README-ned-settings.md) + * [README-rebuild](accedian-skylight_rc/README-rebuild.md) + * [README v3.0.4 2025-08-07](accedian-skylight_rc/README.md) + +* accedian-spp + * [README-ned-settings](accedian-spp/README-ned-settings.md) + * [README v1.7 2025-07-23](accedian-spp/README.md) + +* actelis-ead + * [README-ned-settings](actelis-ead/README-ned-settings.md) + * [README v1.0.8 2025-01-16](actelis-ead/README.md) + +* adtran-dpoe + * [README-ned-settings](adtran-dpoe/README-ned-settings.md) + * [README v1.0.2 2025-08-11](adtran-dpoe/README.md) + +* adva-825 + * [README-ned-settings](adva-825/README-ned-settings.md) + * [README v4.1.22 2025-07-02](adva-825/README.md) + +* alu-isam + * [README-ned-settings](alu-isam/README-ned-settings.md) + * [README v1.5.2 2025-11-11](alu-isam/README.md) + +* alu-omniswitch-6k + * [README v2.5.7 2024-10-03](alu-omniswitch-6k/README.md) + +* alu-sr + * [README-ned-settings](alu-sr/README-ned-settings.md) + * [README v8.65.2 2025-11-11](alu-sr/README.md) + +* arista-dcs + * [README-ned-settings](arista-dcs/README-ned-settings.md) + * [README v5.30.7 2025-10-31](arista-dcs/README.md) + +* arris-cmts + * [README-ned-settings](arris-cmts/README-ned-settings.md) + * [README v1.10.11 2025-08-26](arris-cmts/README.md) + +* brocade-ironware + * [README-ned-settings](brocade-ironware/README-ned-settings.md) + * [README v4.2.4 2024-10-03](brocade-ironware/README.md) + +* casa-ccap + * [README-ned-settings](casa-ccap/README-ned-settings.md) + * [README v1.4.10 2024-09-24](casa-ccap/README.md) + +* ceragon-ip20 + * [README-ned-settings](ceragon-ip20/README-ned-settings.md) + * [README v1.10.1 2025-10-02](ceragon-ip20/README.md) + +* checkpoint-gaiaos_rest + * [README-ned-settings](checkpoint-gaiaos_rest/README-ned-settings.md) + * [README v1.11.4 2025-04-04](checkpoint-gaiaos_rest/README.md) + +* ciena-acos + * [README-ned-settings](ciena-acos/README-ned-settings.md) + * [README v6.6.3 2025-06-05](ciena-acos/README.md) + +* ciena-mcp + * [README-ned-settings](ciena-mcp/README-ned-settings.md) + * [README.TSM](ciena-mcp/README.TSM.md) + * [README v1.9.22 2025-07-29](ciena-mcp/README.md) + +* ciena-saos_nc + * [README-ned-settings](ciena-saos_nc/README-ned-settings.md) + * [README-rebuild](ciena-saos_nc/README-rebuild.md) + * [README v1.1 2025-10-08](ciena-saos_nc/README.md) + +* cisco-aireos + * [README-ned-settings](cisco-aireos/README-ned-settings.md) + * [README v3.9.26 2025-08-15](cisco-aireos/README.md) + +* cisco-apicdc + * [README-ned-settings](cisco-apicdc/README-ned-settings.md) + * [README v3.21.3 2025-08-29](cisco-apicdc/README.md) + +* cisco-asa + * [README-ned-settings](cisco-asa/README-ned-settings.md) + * [README v6.18.29 2025-08-27](cisco-asa/README.md) + +* cisco-cnc_rc + * [README-ned-settings](cisco-cnc_rc/README-ned-settings.md) + * [README-rebuild](cisco-cnc_rc/README-rebuild.md) + * [README v1.0.10 2025-08-15](cisco-cnc_rc/README.md) + +* cisco-esa + * [README-ned-settings](cisco-esa/README-ned-settings.md) + * [README v2.0.12 2025-08-13](cisco-esa/README.md) + +* cisco-fmc + * [README-ned-settings](cisco-fmc/README-ned-settings.md) + * [README v1.7.3 2025-11-07](cisco-fmc/README.md) + +* cisco-ftd + * [README-ned-settings](cisco-ftd/README-ned-settings.md) + * [README v1.11.10 2024-10-21](cisco-ftd/README.md) + +* cisco-fxos + * [README-ned-settings](cisco-fxos/README-ned-settings.md) + * [README v1.1.13 2025-09-26](cisco-fxos/README.md) + +* cisco-ios + * [README-ned-settings](cisco-ios/README-ned-settings.md) + * [README v6.110.4 2025-11-04](cisco-ios/README.md) + +* cisco-iosxr + * [README-ned-settings](cisco-iosxr/README-ned-settings.md) + * [README v7.74.5 2025-11-11](cisco-iosxr/README.md) + +* cisco-iosxr_gnmi + * [README-ned-settings](cisco-iosxr_gnmi/README-ned-settings.md) + * [README-rebuild](cisco-iosxr_gnmi/README-rebuild.md) + * [README v1.1.12 2025-11-05](cisco-iosxr_gnmi/README.md) + +* cisco-iosxr_nc + * [README-ned-settings](cisco-iosxr_nc/README-ned-settings.md) + * [README-rebuild](cisco-iosxr_nc/README-rebuild.md) + * [README v1.1.1 2025-10-21](cisco-iosxr_nc/README.md) + +* cisco-iosxr_netconf + * [README v25.2.1 2025-06-30](cisco-iosxr_netconf/README.md) + +* cisco-ise + * [README-ned-settings](cisco-ise/README-ned-settings.md) + * [README v1.1.2 2024-08-29](cisco-ise/README.md) + +* cisco-nx + * [README-ned-settings](cisco-nx/README-ned-settings.md) + * [README v5.31 2025-10-28](cisco-nx/README.md) + +* cisco-sma + * [README-ned-settings](cisco-sma/README-ned-settings.md) + * [README v2.1.2 2025-08-13](cisco-sma/README.md) + +* cisco-staros + * [README-ned-settings](cisco-staros/README-ned-settings.md) + * [README v5.57.5 2025-11-07](cisco-staros/README.md) + +* citrix-netscaler + * [README-ned-settings](citrix-netscaler/README-ned-settings.md) + * [README v4.5.12 2025-02-21](citrix-netscaler/README.md) + +* eci-muse + * [README-ned-settings](eci-muse/README-ned-settings.md) + * [README v1.7.3 2025-11-04](eci-muse/README.md) + +* ericsson-efn324 + * [README-ned-settings](ericsson-efn324/README-ned-settings.md) + * [README v2.1.7 2025-07-14](ericsson-efn324/README.md) + +* ericsson-enm + * [README-ned-settings](ericsson-enm/README-ned-settings.md) + * [README v1.1.1 2025-07-15](ericsson-enm/README.md) + +* ericsson-minilink6352 + * [README-ned-settings](ericsson-minilink6352/README-ned-settings.md) + * [README v1.2.4 2025-03-19](ericsson-minilink6352/README.md) + +* ericsson-minilink6600 + * [README-ned-settings](ericsson-minilink6600/README-ned-settings.md) + * [README v1.3.0 2025-03-25](ericsson-minilink6600/README.md) + +* etsi-sol003 + * [README-ned-settings](etsi-sol003/README-ned-settings.md) + * [README v1.13.20 2025-04-07](etsi-sol003/README.md) + +* extreme-xos + * [README-ned-settings](extreme-xos/README-ned-settings.md) + * [README v1.5.5 2024-08-29](extreme-xos/README.md) + +* f5-bigip + * [README-ned-settings](f5-bigip/README-ned-settings.md) + * [README v3.24.5 2025-08-28](f5-bigip/README.md) + +* fireeye-cms + * [README-ned-settings](fireeye-cms/README-ned-settings.md) + * [README v1.0.6 2024-08-23](fireeye-cms/README.md) + +* fortinet-fmg + * [README-ned-settings](fortinet-fmg/README-ned-settings.md) + * [README v4.3.40 2025-10-31](fortinet-fmg/README.md) + +* fortinet-fortios + * [README-ned-settings](fortinet-fortios/README-ned-settings.md) + * [README v5.11.27 2025-11-07](fortinet-fortios/README.md) + +* furukawa-fx1 + * [README-ned-settings](furukawa-fx1/README-ned-settings.md) + * [README v2.0.5 2025-08-22](furukawa-fx1/README.md) + +* hpe-ihss + * [README-ned-settings](hpe-ihss/README-ned-settings.md) + * [README v1.2.7.1 2024-09-02](hpe-ihss/README.md) + +* huawei-ias + * [README-ned-settings](huawei-ias/README-ned-settings.md) + * [README v2.9 2025-11-07](huawei-ias/README.md) + +* huawei-imanager + * [README-ned-settings](huawei-imanager/README-ned-settings.md) + * [README v1.3.15 2024-12-02](huawei-imanager/README.md) + +* huawei-imanagertl1 + * [README-ned-settings](huawei-imanagertl1/README-ned-settings.md) + * [README v1.7.10 2024-10-03](huawei-imanagertl1/README.md) + +* huawei-nce + * [README-ned-settings](huawei-nce/README-ned-settings.md) + * [README v1.0.28 2025-10-02](huawei-nce/README.md) + +* huawei-vrp + * [README-ned-settings](huawei-vrp/README-ned-settings.md) + * [README v6.81.1 2025-11-12](huawei-vrp/README.md) + +* huawei-vrp_nc + * [README-ned-settings](huawei-vrp_nc/README-ned-settings.md) + * [README-rebuild](huawei-vrp_nc/README-rebuild.md) + * [README v1.2 2025-10-08](huawei-vrp_nc/README.md) + +* infoblox-nios + * [README-ned-settings](infoblox-nios/README-ned-settings.md) + * [README v4.0.11 2024-10-01](infoblox-nios/README.md) + +* juniper-junos + * [README v4.18.27 2025-11-06](juniper-junos/README.md) + +* juniper-junos_nc + * [README-ned-settings](juniper-junos_nc/README-ned-settings.md) + * [README-rebuild](juniper-junos_nc/README-rebuild.md) + * [README v1.1.27 2025-11-04](juniper-junos_nc/README.md) + +* mrv-masteros + * [README-ned-settings](mrv-masteros/README-ned-settings.md) + * [README v3.8.19 2025-07-08](mrv-masteros/README.md) + +* nec-ipasolink-vr + * [README-ned-settings](nec-ipasolink-vr/README-ned-settings.md) + * [README v1.0.0 2025-02-03](nec-ipasolink-vr/README.md) + +* nokia-apc + * [README-ned-settings](nokia-apc/README-ned-settings.md) + * [README v1.0.10 2024-10-02](nokia-apc/README.md) + +* nokia-srlinux_gnmi + * [README-ned-settings](nokia-srlinux_gnmi/README-ned-settings.md) + * [README-rebuild](nokia-srlinux_gnmi/README-rebuild.md) + * [README v1.2.15 2025-08-15](nokia-srlinux_gnmi/README.md) + +* nokia-sros_nc + * [README-ned-settings](nokia-sros_nc/README-ned-settings.md) + * [README-rebuild](nokia-sros_nc/README-rebuild.md) + * [README v1.0.29 2025-11-05](nokia-sros_nc/README.md) + +* oneaccess-oneos + * [README-ned-settings](oneaccess-oneos/README-ned-settings.md) + * [README v3.4.11 2025-06-12](oneaccess-oneos/README.md) + +* onf-tapi_rc + * [README-ned-settings](onf-tapi_rc/README-ned-settings.md) + * [README-rebuild](onf-tapi_rc/README-rebuild.md) + * [README v2.0.51 2025-11-05](onf-tapi_rc/README.md) + +* openstack-cos + * [README-ned-settings](openstack-cos/README-ned-settings.md) + * [README v4.2.35 2025-05-09](openstack-cos/README.md) + +* overture-1400 + * [README-ned-settings](overture-1400/README-ned-settings.md) + * [README v4.1.6 2025-06-05](overture-1400/README.md) + +* paloalto-panos_cli + * [README-ned-settings](paloalto-panos_cli/README-ned-settings.md) + * [README v4.11.17 2025-09-25](paloalto-panos_cli/README.md) + +* pica8-picos + * [README-ned-settings](pica8-picos/README-ned-settings.md) + * [README v1.4.10 2025-03-07](pica8-picos/README.md) + +* proxmox-ve + * [README-ned-settings](proxmox-ve/README-ned-settings.md) + * [README v1.0.5 2024-08-29](proxmox-ve/README.md) + +* quagga-bgp + * [README-ned-settings](quagga-bgp/README-ned-settings.md) + * [README v4.2.15 2025-09-01](quagga-bgp/README.md) + +* rad-vx + * [README-ned-settings](rad-vx/README-ned-settings.md) + * [README v1.18.16 2025-01-24](rad-vx/README.md) + +* radware-cc + * [README v1.0.0.1 2025-09-26](radware-cc/README.md) + +* redback-se + * [README-ned-settings](redback-se/README-ned-settings.md) + * [README v1.6 2025-10-22](redback-se/README.md) + +* redhat-ansible + * [README-ned-settings](redhat-ansible/README-ned-settings.md) + * [README v1.0.13 2024-09-06](redhat-ansible/README.md) + +* redhat-dir389 + * [README-ned-settings](redhat-dir389/README-ned-settings.md) + * [README v1.2.6 2025-01-07](redhat-dir389/README.md) + +* riverbed-steelhead + * [README-ned-settings](riverbed-steelhead/README-ned-settings.md) + * [README v4.0.4.1 2025-09-25](riverbed-steelhead/README.md) + +* sfr-nbe300 + * [README-ned-settings](sfr-nbe300/README-ned-settings.md) + * [README v2.2.4 2025-03-07](sfr-nbe300/README.md) + +* siae-smdc_rc + * [README-ned-settings](siae-smdc_rc/README-ned-settings.md) + * [README-rebuild](siae-smdc_rc/README-rebuild.md) + * [README v1.0.16 2025-07-02](siae-smdc_rc/README.md) + +* tejas-nms5500 + * [README-ned-settings](tejas-nms5500/README-ned-settings.md) + * [README v1.0.7 2024-08-29](tejas-nms5500/README.md) + +* tilgin-tgem + * [README-ned-settings](tilgin-tgem/README-ned-settings.md) + * [README v1.0.1 2025-04-17](tilgin-tgem/README.md) + +* unix-bind + * [README-ned-settings](unix-bind/README-ned-settings.md) + * [README v2.2.1 2025-07-08](unix-bind/README.md) + +* vecima-eac + * [README-ned-settings](vecima-eac/README-ned-settings.md) + * [README v1.0.2 2025-01-07](vecima-eac/README.md) + +* vecima-rpd + * [README-ned-settings](vecima-rpd/README-ned-settings.md) + * [README v1.0.2 2025-10-14](vecima-rpd/README.md) + +* viptela-vmanage + * [README-ned-settings](viptela-vmanage/README-ned-settings.md) + * [README v1.6.28 2025-09-03](viptela-vmanage/README.md) + +* vmware-vsphere + * [README-ned-settings](vmware-vsphere/README-ned-settings.md) + * [README v3.3.18 2025-03-03](vmware-vsphere/README.md) + +* zte-xpon + * [README-ned-settings](zte-xpon/README-ned-settings.md) + * [README v4.4.5 2025-11-07](zte-xpon/README.md) + +* zte-zxros + * [README-ned-settings](zte-zxros/README-ned-settings.md) + * [README v1.2.6 2025-07-07](zte-zxros/README.md) + diff --git a/a10-acos/README-ned-settings.md b/a10-acos/README-ned-settings.md new file mode 100644 index 00000000..2dbb5444 --- /dev/null +++ b/a10-acos/README-ned-settings.md @@ -0,0 +1,397 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/a10-acos/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/a10-acos/ + device + /ncs:/device/devices/device:/ned-settings/a10-acos/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings a10-acos + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings a10-acos + 2. a10-connection-settings + 3. live-status + 4. connection + 5. proxy + 6. developer + 7. rpc-actions + 7.1. expect-patterns + 8. dynamic-errors + 9. logger + 10. transaction + ``` + + +# 1. ned-settings a10-acos +-------------------------- + + a10-acos device specific NED settings. + + + - a10-acos aflex-scripts-support (default disabled) + + Enable this ned-settings in order to fetch and manage (create/update/delete) aFlex scripts. By + default, this feature is disabled. + + enabled - enabled. + + disabled - disabled. + + + - a10-acos trans-id-method (default config-hash) + + Configure how the NED shall calculate the transaction id. Typically used after each commit and + for check-sync operations. + + config-hash - Use a snapshot of the running config for calculation.(default). + + last-modified-timestamp - Use the 'time last modified' time stamp generated by the device for + calculation. Note, this time stamp is not available on all devices. + See README. + + last-saved-timestamp - Use the 'time last saved' time stamp generated by the device for + calculation. Note, this method is not reliable. See README. + + + - a10-acos a10-active-partition + + Active partition. + + + - a10-acos a10-abort-when-config-session-exist (default false) + + Active partition. + + + - a10-acos a10-write-memory-all-partitions (default true) + + Set to true if the device supports write memory all-partitions. + + + - a10-acos extended-parser (default auto) + + Make the a10-acos NED handle CLI parsing (i.e. transform the running-config from the device to + the model based config tree). + + disabled - Load configuration the standard way. + + robust-mode - The configuration dump is run through a pre-parser which is cleaning it + from all elements currently not supported in the YANG model (default). + + turbo-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO using a + Maapi SetValues() call. + + turbo-xml-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO in XML + format. + + old-robust-mode - Makes the NED alter the config dump such that all mode switches are always + done from top and down instead of from below and up (with the 'exit' + command) before given to the NCS/NSO parser. +The number of lines in the + config dump will increase a lot with this feature enabled. (default). + + auto - Uses turbo-mode when available, will use fastest availablemethod to load + data to NSO. If NSO doesn't support data-loading from CLI NED, robust-mode + is used. + + + - a10-acos partial-show-method (default full-config) + + Configure partial show method execution. + + full-config - Fetch full configuration from the device and filter the config. This method + should be used for devices not supporting partial show commands, eg show + running-config access-list 100. + + partial-config - Sends partial show commands to the device to fetch onlythe needed part of + the config. + + + - a10-acos shared-partition-mode (default in-config) + + Set the shared partition location: directly under config or under partition-config. + + in-partition-config - Shared partition placed under partition-config. + + in-config - Shared partition directly under root config. + + +# 2. ned-settings a10-acos a10-connection-settings +-------------------------------------------------- + + Configure settings specific to the connection between NED and device. + + + - a10-connection-settings device-output-delay (default 0) + + Delay in milliseconds after each config command output to the device. + + +# 3. ned-settings a10-acos live-status +-------------------------------------- + + Configure NED settings related to live-status. + + + - live-status time-to-live (default 50) + + Define time-to-live for data fetched from the device via live-status.(default 50). + + +# 4. ned-settings a10-acos connection +------------------------------------- + + Connection configuration. + + + - connection ssh client (default ganymed) + + Configure the SSH client to use. Relevant only when using the NED with NSO 5.6 or later. + + ganymed - The legacy SSH client. Used on all older versions of NSO. + + sshj - The new SSH client with support for the latest crypto features. This is the default + when using the NED on NSO 5.6 or later. + + + - connection ssh host-key known-hosts-file + + Path to openssh formatted 'known_hosts' file containing valid host keys. + + + - connection ssh host-key public-key-file + + Path to openssh formatted public (.pub) host key file. + + + - connection ssh auth-key private-key-file + + Path to openssh formatted private key file. + + + - connection number-of-retries (default 0) + + Configure max number of extra retries the NED will try to connect to the device before giving + up. Default 0. + + + - connection time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + + - connection send-login-newline (default false) + + +# 5. ned-settings a10-acos proxy +-------------------------------- + + Configure NED to access device via a proxy. + + + - proxy remote-connection + + Connection type between proxy and device. + + ssh - ssh. + + telnet - telnet. + + serial - serial. + + + - proxy remote-address + + Address of host behind the proxy. + + + - proxy remote-port + + Port of host behind the proxy. + + + - proxy remote-name + + User name on the device behind the proxy. + + + - proxy remote-password + + Password on the device behind the proxy. + + + - proxy remote-secondary-password + + Password on the device behind the proxy. + + + - proxy proxy-prompt + + Prompt pattern on the proxy host. + + + - proxy remote-ssh-args + + Additional arguments used to establish proxy connection. + + +# 6. ned-settings a10-acos developer +------------------------------------ + + Contains settings used by the NED developers. + + + - developer load-from-file + + Make the NED load a file containing raw device config when doing sync-from. Does only work on + NETSIM targets. + + + - developer load-native-config allow-delete (default false) + + Enable this setting to be able to handle limited delete operations with 'load-native-config'. + Please note that not all syntax available on a real device works, some delete operations can + not be parsed by the NED. Use the 'verbose' flag to 'load-native-config' to see if delete + commands can be parsed. Currently this is only supported when 'extended-parser' is set to + 'turbo-xml-mode'. + + + - developer load-native-config delete-with-remove (default false) + + Enable this setting to use 'remove' instead of 'delete' when sending delete operations to NSO. + This is useful when doing delete commands for data that might not be present in CDB. Please + note that deletes for missing data will still be part of transaction, and will be sent to + device. Use with care, and do proper testing to understand behaviour. + + + - developer trace-connection (default false) + + Enable developer connection tracing. WARNING: may choke NSO with large printouts. + + + - developer platform model + + Override device model name/number. + + + - developer platform name + + Override device name. + + + - developer platform version + + Override device version. + + +# 7. ned-settings a10-acos rpc-actions +-------------------------------------- + + RPC actions related configurations. + + +## 7.1. ned-settings a10-acos rpc-actions expect-patterns +--------------------------------------------------------- + + List of expected patterns and prompts when executing commands. It can be used to define custom + expected patterns, for example to wait for a number of characters (eg .....) in order to implement + an automatic time-out reset mechanism. NOTE: the patterns represent regular expressions. + + - rpc-actions expect-patterns + + - pattern + + +# 8. ned-settings a10-acos dynamic-errors +----------------------------------------- + + List of device errors. The NED will throw error when it encounter a message from this list. + + - a10-acos dynamic-errors + + - error + + +# 9. ned-settings a10-acos logger +--------------------------------- + + Settings for controlling logs generated. + + + - logger level (default info) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - logger java (default true) + + Toggle logs to be added to ncs-java-vm.log. + + +# 10. ned-settings a10-acos transaction +--------------------------------------- + + + - transaction cleartext-provisioning (default enabled) + + Enable this to allow for cleartext key/passwords provisioning without going out-of-sync(i.e. + where device obfuscates/encrypts value in running-config). + + enabled - enabled. + + disabled - disabled. + + + - transaction cleartext-stored-encrypted (default disabled) + + When 'cleartext-provisioning' is enabled, enable this setting to enforce keys/passwords CDB + storedvalues to be encrypted using NSO's built in encryption types (e.g. + 'tailf:aes-cfb-128-encrypted-string').NOTE: the type of the values in the yang-model of alu-sr + is NOT the encrypted type by default, it is still plain 'string'. However, the service + template/code used to set the values must use an encrypted type.The NED can be instructed to + use tailf:aes-cfb-128-encrypted-string for passwords by default and hence to do + auto-encryption of the passwords, but this requires to recompile the NED with + NEDCOM_SECRET_TYPE flag set (eg 'make NEDCOM_SECRET_TYPE="tailf:aes-cfb-128-encrypted-string" + clean all'). + + enabled - enabled. + + disabled - disabled. + + diff --git a/a10-acos/README.md b/a10-acos/README.md new file mode 100644 index 00000000..80e138b6 --- /dev/null +++ b/a10-acos/README.md @@ -0,0 +1,1109 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + 11. Aflex scripts + 12. NED Secrets - Securing your Secrets + ``` + + +# 1. General +------------ + + This document describes the a10-acos NED. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | Default emulated device: AX Series Advanced Traffic Manager | + | | | v2.6.1-GR1 | + | | | | + | check-sync | yes | check-sing using trans-id | + | | | | + | partial-sync-from | yes | This feature is supported by filtering the needed config from a | + | | | full show (device does not support partial show) | + | | | | + | live-status actions | yes | Supports most of the device needed actions | + | | | | + | live-status show | no | The NED does not implement TTL-based data | + | | | | + | load-native-config | no | | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + Custom NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | device partitions | yes | The NED supports device partitions by using config active- | + | | | partition list | + | | | | + | ned-secrets | yes | NED supports device-enctypted password caching. Please check | + | | | README.md | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Thunder Series TH3030S | 2.7.1-GR1 | ACOS | Hardware device | + | (th3030s-1) | | | | + | | | | | + | Thunder Series TH3030S | 2.7.2-P7-SP3 | ACOS | Hardware device | + | (th3030s-2) | | | | + | | | | | + | Thunder Series Unified | 5.2.0 | ACOS | Virtual device | + | Application Service | | | | + | Gateway vThunder | | | | + | (vThunder-1) | | | | + | | | | | + | Thunder Series Unified | 5.2.0 | ACOS | Virtual device | + | Application Service | | | | + | Gateway vThunder | | | | + | (vThunder-2) | | | | + | | | | | + | Thunder Series Unified | 2.7.2-P4-SP1 | ACOS | Virtual device | + | Application Service | | | | + | Gateway vThunder | | | | + | (vThunder-27) | | | | + | | | | | + | AX Series Advanced | 2.7.2-P10 | ACOS | Virtual device | + | Traffic Manager vThunder | | | | + | (vThunder-7) | | | | + | | | | | + | Thunder Series Unified | 4.1.4-GR1-P1 | ACOS | Virtual device | + | Application Service | | | | + | Gateway vThunder | | | | + | (vThunder-8) | | | | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--a10-acos-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-a10-acos-1.0.1.signed.bin + > ./ncs-6.0-a10-acos-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-a10-acos-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-a10-acos-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `a10-acos-.`: + + ``` + > tar xfz ncs-6.0-a10-acos-1.0.1.tar.gz + > ls -d */ + a10-acos-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package a10-acos-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package a10-acos-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-a10-acos-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package a10-acos-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/a10-acos-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-a10-acos-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-a10-acos-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install a10-acos-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-a10-acos-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-a10-acos-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id a10-acos-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-a10-acos-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings a10-acos logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings a10-acos logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.a10 \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings a10-acos logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings a10-acos logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + The following is an example of configuration data (CLI NED commands) that can be sent to an a10-acos device: + ``` + system resource-usage class-list-ipv6-addr-count 600000 + system per-vlan-limit mcast 10000 + system per-vlan-limit unknown-ucast 10000 + + snmp-server enable traps slb application-buffer-limit + snmp-server enable traps slb server-conn-limit + snmp-server enable traps slb server-conn-resume + snmp-server enable traps slb server-down + snmp-server enable traps slb server-up + snmp-server enable traps slb service-conn-limit + snmp-server enable traps slb service-conn-resume + snmp-server enable traps slb service-down + snmp-server enable traps slb service-up + snmp-server enable traps slb vip-connlimit + snmp-server enable traps slb vip-connratelimit + snmp-server enable traps slb vip-port-connlimit + snmp-server enable traps slb vip-port-connratelimit + snmp-server enable traps slb vip-port-down + snmp-server enable traps slb vip-port-up + snmp-server enable traps system control-cpu-high + snmp-server enable traps system fan + snmp-server enable traps system high-disk-use + snmp-server enable traps system high-memory-use + snmp-server enable traps system high-temp + snmp-server enable traps system pri-disk + snmp-server enable traps system sec-disk + snmp-server enable traps system shutdown + snmp-server enable traps system start + snmp-server enable traps vrrp-a active + snmp-server enable traps vrrp-a standby + + access-list 100 permit ip any any + + interface ethernet 1 + l3-vlan-fwd-disable + load-interval 200 + enable + icmp-rate-limit 1000 lockup 2000 200 + access-list 100 in + ip cache-spoofing-port + ip helper-address 2.2.2.1 + ip helper-address 2.2.3.1 + ip nat inside + ip nat outside + ip router isis ASD + ipv6 address fe01::/64 anycast + ipv6 address fe02::/64 anycast + ipv6 nat inside + ipv6 nat outside + + interface ethernet 2 + icmp-rate-limit 1000 + + slb server SERVER-COVERAGE-1 1.1.1.1 + slb server SERVER-COVERAGE-2 2.1.1.1 + + slb service-group SERV-GROUP-COVERAGE-1 tcp + slb service-group SERV-GROUP-COVERAGE-2 tcp + slb service-group SERV-GROUP-COVERAGE-3 tcp + slb service-group SERV-GROUP-COVERAGE-4 tcp + slb service-group SERV-GROUP-COVERAGE-5 tcp + slb service-group SERV-GROUP-COVERAGE-6 tcp + slb service-group SERV-GROUP-COVERAGE-7 tcp + slb service-group SERV-GROUP-COVERAGE-8 tcp + slb service-group SERV-GROUP-COVERAGE-9 tcp + slb service-group SERV-GROUP-COVERAGE-10 tcp + slb service-group SERV-GROUP-COVERAGE-11 tcp + slb service-group SERV-GROUP-COVERAGE-12 tcp + slb service-group SERV-GROUP-COVERAGE-13 tcp + slb service-group SERV-GROUP-COVERAGE-14 tcp + slb service-group SERV-GROUP-COVERAGE-15 tcp + slb service-group SERV-GROUP-COVERAGE-16 tcp + slb service-group SERV-GROUP-COVERAGE-17 tcp + + slb template persist cookie SLB-PERSIST-COOKIE-1 + slb template persist cookie SLB-PERSIST-COOKIE-2 + slb template persist cookie SLB-PERSIST-COOKIE-3 + slb template persist source-ip SLB-PERSIST-SRCIP-1 + slb template persist source-ip SLB-PERSIST-SRCIP-2 + slb template persist source-ip SLB-PERSIST-SRCIP-3 + slb template persist source-ip SLB-PERSIST-SRCIP-4 + slb template persist source-ip SLB-PERSIST-SRCIP-5 + slb template port SLB-TMP-PORT-1 + slb template port SLB-TMP-PORT-2 + slb template port SLB-TMP-PORT-3 + slb template port SLB-TMP-PORT-4 + + slb virtual-server SLB-VIRT-SERVER-COVERAGE-1 5.1.1.1 + slb virtual-server SLB-VIRT-SERVER-COVERAGE-2 5.2.1.1 + slb virtual-server SLB-VIRT-SERVER-COVERAGE-3 5.3.1.1 /24 + slb virtual-server SLB-VIRT-SERVER-COVERAGE-4 5.4.1.1 + slb virtual-server SLB-VIRT-SERVER-COVERAGE-5 5.5.1.1 + + health monitor COVERAGE-HEALTH-1 + retry 5 + up-retry 5 + override-ipv4 1.2.3.5 + override-ipv6 fe01::1 + override-port 1000 + method https expect ASDASD + + health monitor COVERAGE-HEALTH-2 + method https expect response-code 100 + + health monitor COVERAGE-HEALTH-3 + method tcp port 1 send ASDASD response contains ASDASD + ``` + + +# 5. Built in live-status actions +--------------------------------- + + There are two main categories of commands that can be sent using a10-acos NED: configuration commands (RPC's that are sent from the device configuration) and privileged commands (RPC's that are sent from privileged mode). + + Following RPC's exemplify the two categories: + ``` + top devices device a10-0 config config-actions action { action-payload "show running-config" } + top devices device a10-0 live-status exec nonconfig-actions action { action-payload "show interface" } + ``` + + Each main RPC category also divides in the following sub-categories: simple commands that does not use additional prompts (eg "show configuration"), interactive commands that uses additional prompts (eg a RPC's that requests username/password or any other prompts) and internal NED commands, that does not interact with the device but with the NED. + + The last category is supported but not implemented for a specific feature. + Simple command format is as follows: + ``` + action { action-payload "RPC CLI command" } + ``` + + Interactive command contains the simple command and adds the following list: + ``` + action { action-payload "import bw-list bl1 use-mgmt-port scp://11.11.11.11/file" interaction { prompt-pattern "User name.*" value myuser } interaction { prompt-pattern Password.* value mypassword } interaction { prompt-pattern "Do you want to overwrite.*" value yes } interaction { prompt-pattern \"Do you want to save the remote host information.*\" value no } } + ``` + + In the above command, the interaction list defines each prompt that it is expected from the device, along with its corresponding value. Note that prompt-pattern is compiled in a regular expression, so special characters that are expected from prompts should be escaped. + The order of the interaction list definition is not important, but all the possible expected prompts should be defined. For example, in the above command, the prompt "Do you want to overwrite.*" is only active when the file exists. + + Internal commands looks the same as simple commands, but also contain the keyword "internal": + ``` + action { action-payload "Internal RPC" internal } + ``` + + All the above command sub-categories can be chained and sent in the same request, by defining a list of actions: + ``` + top devices device a10-0 live-status exec nonconfig-actions action { action-payload "show route" } action { action-payload "show interfaces" } action { action-payload "Internal RPC" internal } + ``` + + The new implementation of RPC action execution allows a more structured way of sending multiple RPC's, by using its XML format: + ``` + + + import bw-list bl1 use-mgmt-port scp://11.11.11.11/file + + User name.* + admin + + + Password.* + admin + + + Do you want to overwrite.* + yes + + + Do you want to save the remote host information.* + no + + + + show running + + + ``` + + +# 6. Built in live-status show +------------------------------ + + The NED does not support TTL-based live-status data + + +# 7. Limitations +---------------- + + The NED CLI does not implement the device behavior 1 to 1, since the device is not a netconf-compatible device. Also, the NED may use yang model workarounds (like node alternative naming) in order to support device behavior. + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings a10-acos logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings a10-acos logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` + +# 11. Aflex scripts +------------------- + +In order to activate aflex scripts support, run the following command: + +``` +admin@ncs(config-config)# config-actions action { action-payload "running-config display aflex" } +``` +Note: after running this command, it is mandatory to do a sync-from, since the device will introduce the aflex section in the running-config + +Once the aflex is activated, aflex scripts can be managed. +Note: since the device aflex scripts are multi-liners, the script payload needs to be properly escaped by replacing newlines - eg '\n' character with '\\n'. Also double quote - eg " needs to be escaped: '"' -> '\"' +Also, every script should end in a '\n' +Note2: since aflex script is a type string leaf in the NED, the script content needs to be quoted: +``` +admin@ncs(config-config)# aflex test2 +admin@ncs(config-aflex-test2)# script "script-content\n" +``` + +For example, to create a new script that would look like this on the device: +``` +vThunder(NOLICENSE)#show aflex test4 +Name: test4 +Syntax: Check +Virtual port: No +Content: +when HTTP_RESPONSE { + # Check Content-Data to avoid unnecessary collects + if { [HTTP::header "Content-Type"] contains "text" } { + HTTP::collect + } +} +``` +the equivalent config on the NED will look like this: +``` +admin@ncs(config-config)# aflex test2 +admin@ncs(config-aflex-test2)# script "when HTTP_RESPONSE {\n # Check Content-Data to avoid unnecessary collects\n if { [HTTP::header \"Content-Type\"] contains \"text\" } {\n HTTP::collect\n }\n}\n" +admin@ncs(config-aflex-test2)# commit dry-run outformat native +native { + device { + name vThunder-8 + data aflex create test2 + when HTTP_RESPONSE { + # Check Content-Data to avoid unnecessary collects + if { [HTTP::header "Content-Type"] contains "text" } { + HTTP::collect + } + } + . + } +``` + +Disabling aflex support is done by the following command: +``` +admin@ncs(config-config)# config-actions action { action-payload "no running-config display aflex" } +``` + +# 12. NED Secrets - Securing your Secrets +----------------------------------------- + + It is best practice to avoid storing your secrets (e.g. passwords and + shared keys) in plain-text, either on NSO or on the device. In NSO we + support multiple encrypted datatypes that are encrypted using a local + key, similarly many devices such as ALU-SR supports automatically + encrypting all passwords stored on the device. + + Naturally, for security reasons, NSO in general has no way of + encrypting/decrypting passwords with the secret key on the + device. This means that if nothing is done about this we will + become out of sync once we write secrets to the device. + + In order to avoid becoming out of sync the NED reads back these elements + immediately after set and stores the encrypted value(s) in a special + `secrets` table in oper data. Later on, when config is read from the + device, the NED replaces all cached encrypted values with their plaintext + values; effectively avoiding all config diffs in this area. If the values + are changed on the device, the new encrypted value will not match the + cached pair and no replacement will take place. This is desired, since out + of band changes should be detected. + + This handles the device-side encryption, but passwords are still unencrypted + in NSO. To deal with this we support using NSO-encrypted strings instead of + plaintext passwords in the NSO data model. + + --- Handling auto-encryption + + Let us say that we have password-encryption on and we want to write a new + user to our device: + + system + security + user admin + access console + console + member administrative + exit + password + + this will be automatically encrypted by the device + + *A:VSR-7750>config# system security user "admin" + *A:VSR-7750>config>system>security>user# info + ---------------------------------------------- + password "$2y$10$fBSDYG2MHpdpCTDQhq7BE.ojwFR5z10g61PUqWaXb52GXg0Ge8d8W" + access console + console + member "administrative" + exit + ---------------------------------------------- + + But the secrets management will store this new encrypted value in our `secrets` table: + + admin@ncs# show devices device dev-1 ned-settings secrets + ID ENCRYPTED REGEX + --------------------------------------------------------------------------------------------------------------- + /system/security/user_admin_/password/id $2y$10$fBSDYG2MHpdpCTDQhq7BE.ojwFR5z10g61PUqWaXb52GXg0Ge8d8W - + + which means that compare-config or sync-from will not show any + changes and will not result in any updates to CDB". In fact, we can + still see the unencrypted value in the device tree: + + admin@ncs(config-config)# show full sys sec user + devices device dev-1 + config + system + security + user admin + access console + console + member administrative + ! + password + + --- Increasing security with NSO-side encryption + + We have two alternatives, either we can manually encrypt our values using + one of the NSO-encrypted types (e.g `aes-256-cfb-128-encrypted-string`) and + set them to the tree, or we can recompile the NED to always encrypt secrets. + + --- Setting encrypted value + + Let us say we know that the NSO-encrypted string + `$2y$10$7ova9fF/bRe9B9GUtjVpA.w5mfeXJXRHyV0KsSfg4XWE9j3Fcq3Qi`, we + can then set it in the device tree as normal + + admin@ncs(config-config)# system security user admin password $2y$10$7ova9fF/bRe9B9GUtjVpA.w5mfeXJXRHyV0KsSfg4XWE9j3Fcq3Qi + admin@ncs(config-config)# commit + + when commiting this value it will be decrypted and the plaintext will be written to the device. + Unlike the previous example the plaintext is not visible in the device tree: + + admin@ncs(config-config)# show full sys sec user + devices device dev-1 + config + system + security + user admin + access console + console + member administrative + ! + password $2y$10$7ova9fF/bRe9B9GUtjVpA.w5mfeXJXRHyV0KsSfg4XWE9j3Fcq3Qi + + On the device side this plaintext value is of course encrypted + with the device key, and just as before we store it in our + `secrets` table: + + admin@ncs# show devices device dev-1 ned-settings secrets + ID ENCRYPTED REGEX + --------------------------------------------------------------------------------------------------------------- + /system/security/user_admin_/password/id $2y$10$fBSDYG2MHpdpCTDQhq7BE.ojwFR5z10g61PUqWaXb52GXg0Ge8d8W - + + We can see that this corresponds to the value set on the device. + + --- Auto-encrypting passwords in NSO + + To avoid having to pre-encrypt your passwords you can rebuild your NED in your OS + command shell specifying an encrypted type for secrets using a command like: + + yourhost:~/ned-folder$ NEDCOM_SECRET_TYPE="tailf:aes-cfb-128-encrypted-string" make -C src/ clean all + + Or by adding the line `NED_EXTRA_BUILDFLAGS ?= NEDCOM_SECRET_TYPE=tailf:aes-cfb-128-encrypted-string` + in top of the `Makefile` located in /src directory. + + Doing this means that even if the input to a password is a plaintext string, NSO will always + encrypt it, and you will never see plain text secrets in the device tree. + + If we reload our example with the new NED all of the secrets are now encrypted: + + admin@ncs(config-config)# show full sys sec user + devices device dev-1 + config + system + security + user admin + access console + console + member administrative + ! + password $2y$10$7ova9fF/bRe9B9GUtjVpA.w5mfeXJXRHyV0KsSfg4XWE9j3Fcq3Qi diff --git a/accedian-nid/README-ned-settings.md b/accedian-nid/README-ned-settings.md new file mode 100644 index 00000000..c76d9e2c --- /dev/null +++ b/accedian-nid/README-ned-settings.md @@ -0,0 +1,186 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/accedian-nid/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/accedian-nid/ + device + /ncs:/device/devices/device:/ned-settings/accedian-nid/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings accedian-nid + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings accedian-nid + 2. connection + 3. deprecated + 4. logger + 5. transaction + 6. sfp-ports + 7. developer + ``` + + +# 1. ned-settings accedian-nid +------------------------------ + + Configure settings specific to the connection between NED and device. + + + - extended-parser (default auto) + + Make the NED handle CLI parsing (i.e. transform the running-config from the device to the + model based config tree). + + auto - Uses turbo-mode when available, will use fastest availablemethod to load + data to NSO. If NSO doesn't support data-loading from CLI NED, robust-mode + is used. + + turbo-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO using maapi + setvalues call. + + turbo-xml-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO in XML + format. + + robust-mode - Makes the NED filter the configuration so that unmodeled content is removed + before being passed to the NSO CLI-engine. This protects against + configuration ending up at the wrong level when NSO CLI parser fallbacks + (which potentially can cause following config to be skipped). + + +# 2. ned-settings accedian-nid connection +----------------------------------------- + + Configure settings specific to the connection between NED and device. + + + - number-of-retries (default 0) + + Configure max number of retries the NED will try to connect to the device before giving up. + Default 0. + + + - time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + + - connector + + Change the default connector, e.g. 'ned-connector-default.json'. + + +# 3. ned-settings accedian-nid deprecated +----------------------------------------- + + Deprecated ned-settings. + + + - connection legacy-mode (default disabled) + + enabled - enabled. + + disabled - disabled. + + +# 4. ned-settings accedian-nid logger +------------------------------------- + + Settings for controlling logs generated. + + + - level (default info) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - java (default true) + + Toggle logs to be added to ncs-java-vm.log. + + +# 5. ned-settings accedian-nid transaction +------------------------------------------ + + Transaction specific settings. + + + - trans-id-method (default modeled-config) + + Select the method for calculating transaction-id. + + modeled-config - Use a snapshot of the data of only the modeled parts of running config for + calculation. + + full-config - Use a snapshot of the full running config for calculation. + + device-custom - Use a device custom method to get a value to use for trans-id. + + +# 6. ned-settings accedian-nid sfp-ports +---------------------------------------- + + This list contains port names that will keep 'force-tx-on'. If the list is empty 'force-tx-on' + will be left as it is. + + - sfp-ports + + - port + + Port name,case sensitive. + + +# 7. ned-settings accedian-nid developer +---------------------------------------- + + Contains settings used for debugging (intended for NED developers). + + + - progress-verbosity (default debug) + + Maximum NED verbosity level which will get written in devel.log file. + + disabled - disabled. + + normal - normal. + + verbose - verbose. + + very-verbose - very-verbose. + + debug - debug. + + diff --git a/accedian-nid/README.md b/accedian-nid/README.md new file mode 100644 index 00000000..7db43332 --- /dev/null +++ b/accedian-nid/README.md @@ -0,0 +1,700 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + ``` + + +# 1. General +------------ + + This document describes the accedian-nid NED. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | - | + | | | | + | check-sync | yes | - | + | | | | + | partial-sync-from | yes | - | + | | | | + | live-status actions | yes | - | + | | | | + | live-status show | no | - | + | | | | + | load-native-config | no | - | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | accedian-GT | 7.1 | | - | + | | | | | + | accedian-LT | 7.1 | | - | + | | | | | + | accedian-TE | 7.1 | | - | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--accedian-nid-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-accedian-nid-1.0.1.signed.bin + > ./ncs-6.0-accedian-nid-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-accedian-nid-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-accedian-nid-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `accedian-nid-.`: + + ``` + > tar xfz ncs-6.0-accedian-nid-1.0.1.tar.gz + > ls -d */ + accedian-nid-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package accedian-nid-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package accedian-nid-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-accedian-nid-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package accedian-nid-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/accedian-nid-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-accedian-nid-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-accedian-nid-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install accedian-nid-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-accedian-nid-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-accedian-nid-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id accedian-nid-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-accedian-nid-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings accedian-nid logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings accedian-nid logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.accediannid \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings accedian-nid logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings accedian-nid logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + NONE + + +# 5. Built in live-status actions +--------------------------------- + + NONE + + +# 6. Built in live-status show +------------------------------ + + NONE + + +# 7. Limitations +---------------- + + NONE + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings accedian-nid logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings accedian-nid logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` diff --git a/accedian-skylight_rc/README-ned-settings.md b/accedian-skylight_rc/README-ned-settings.md new file mode 100644 index 00000000..082d38a0 --- /dev/null +++ b/accedian-skylight_rc/README-ned-settings.md @@ -0,0 +1,979 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/accedian-skylight_rc/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/accedian-skylight_rc/ + device + /ncs:/device/devices/device:/ned-settings/accedian-skylight_rc/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings accedian-skylight_rc + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings accedian-skylight_rc + 2. connection + 2.1. authentication + 2.1.1. token-request + 2.1.2. token-revoke + 2.2. ssl + 2.2.1. mtls + 3. live-status + 4. restconf + 4.1. cache + 4.2. config + 4.2.1. deviations + 4.3. live-status + 4.4. notif + 4.5. yang-push + 5. logger + 6. general + 6.1. capabilities + 6.2. config + 6.3. live-status + 6.4. notif + ``` + + +# 1. ned-settings accedian-skylight_rc +-------------------------------------- + + +# 2. ned-settings accedian-skylight_rc connection +------------------------------------------------- + + Settings for the RESTCONF connection. + + + - connection use-host-name (default false) + + Configure the NED whether to use the host name or the ip address to the device when + connecting. If set to true the host name will be used if possible. + + + - connection custom-hostname + + The hostname of the device. This is used to connect to the device. + + +## 2.1. ned-settings accedian-skylight_rc connection authentication +------------------------------------------------------------------- + + Authentication related settings. + + + - authentication method (default basic) + + Configure authentication method to use when the NED interacts with the RESTCONF device. + + basic - Use standard 'Basic' authentication. + + none - No additional authentication is done. This option shall for instance be used + on devices that only rely on authentication via mTLS. + + bearer-token - Use a 'bearer token' based authentication. This does require additional + configurations. Either a static token or address info etc to a token broker. + + + - authentication use-token-cache (default false) + + When set to true, the NED will cache the negotiated authentication token for later use in any subsequent connections. + The cache reduces the number of round trips needed when connecting to the target. Applicable token based mechanisms + like the "bearer-token". + The feature do require adaptions of the NED to detect when cached token is regarded as expired by the device, I.e the + NED needs to be instrumented with pattern for typical device replies that indicate "token expired". + Use with caution when NED is interacting with any other device. + + + - authentication mode + + The bearer-token method has the following additional settings. + + probe - Do a dynamic probe for the bearer token to be used via a token broker. This + does require the additional 'token-request' settings to be configured. + + static-token - Use a statically configured token configured in the 'token-value' setting. + + + - authentication token-value + + Configure a static token value. + + +### 2.1.1. ned-settings accedian-skylight_rc connection authentication token-request +------------------------------------------------------------------------------------ + + Bearer token request settings. + + + - token-request url (default /restconf/auth) + + URL path to bearer token broker. This does not use the base-url. Default: /restconf/auth. + + + - token-request port + + Port used used by bearer token broker. Default: use same as restconf connection. + + + - token-request address + + Address used used by bearer token broker. Default: use same as restconf connection. + + + - token-request username-parameter + + Username parameter name if different from 'username' in configured auth group. + + + - token-request password-parameter + + Password parameter name if different from 'password' in configured auth group. + + +### 2.1.2. ned-settings accedian-skylight_rc connection authentication token-revoke +----------------------------------------------------------------------------------- + + Bearer token revoke settings. If configured, the used token will be automatically closed when the + NED is closing. + + + - token-revoke url + + URL path to bearer token broker. This does not use the base-url. + + + - token-revoke port + + Port used used by bearer token broker. Default: use same as for requesting token. + + + - token-revoke address + + Address used used by bearer token broker. Default: use same as for requesting token. + + + - token-revoke query + + Additional query parameter(s) used when doing the token revokation. + + +## 2.2. ned-settings accedian-skylight_rc connection ssl +-------------------------------------------------------- + + Settings related to SSL/TLS enabled connections. + + + - ssl accept-any + + Accept any SSL certificate presented by the device. + Warning! This enables Man in the Middle attacks and should only be used for testing and troubleshooting. + + + - ssl hostname + + Device hostname/fqdn. Useful when SSL certificate CN verification fails because NSO uses IP + address instead of hostname. Note: when accept-any = false and there is no + connection/ssl/certificate defined, the NED will automatically fetch the server certificate. + + + - ssl ciphers + + Configure permitted ciphers to use when doing TLS handshake. Leave empty to use system + default. + + + - ssl protocols + + Configure permitted protocol versions to use when doing TLS handshake. Leave empty to use + system default. + + + - ssl certificate + + Configure a certificate to be used for identifying the device to connect to. It can be either + a host certificate identifying the device or a self signed root certificate that has been used + for signing the certificate on the device. + + SSL certificate stored in DER format but since it is entered as Base64 it is very similar to PEM but + without banners like: + "----- BEGIN CERTIFICATE -----". + + Default uses the default trusted certificates installed in Java JVM. + + An easy way to get the PEM of a server: + openssl s_client -connect HOST:PORT + + +### 2.2.1. ned-settings accedian-skylight_rc connection ssl mtls +---------------------------------------------------------------- + + Settings related to mutual TLS (mTLS) Note, if mTLS is to be used without any further + authentication mechanism, then ned-settings accedian-skylight_rc connection authentication must be + configured to 'none'. + + + - mtls client certificate + + Configure a certificate to be used by the NED in a mutual TLS (mTLS) setup. This certificate + will be used for identifying the NED by the device. + + SSL/TLS certificate stored in DER format but since it is entered as Base64 it is very similar to + PEM but without banners like: + "----- BEGIN CERTIFICATE -----". + + + - mtls client private-key + + Private key stored in DER format but since it is entered as Base64 it is very similar to PEM but + without banners like: + "----- BEGIN PRIVATE KEY -----". + + The private key is stored encrypted in NSO. + + + - mtls client key-password + + Configure a optional password to the private key from the previous step. The password is + stored encrypted in NSO. + + +# 3. ned-settings accedian-skylight_rc live-status +-------------------------------------------------- + + Configure NED settings related to live-status. + + + - live-status time-to-live (default 50) + + Define time-to-live for data fetched from the device via live-status.(default 50). + + +# 4. ned-settings accedian-skylight_rc restconf +----------------------------------------------- + + Settings related to the RESTCONF API. + + + - restconf url-base (default auto) + + Device RESTCONF API URL base. Note: this setting is automatically configured when one of the + pre-set RESTCONF profiles is used. + + + - restconf get ignore-http-status-code <[ ... ]> + + Configure additional HTTP status codes that shall not trigger and error when the + NED checks the device response upon a RESTCONF GET call. By default the NED will + not trigger an HTTP status codes 400 (bad request) and 404 (not found) when trying + to fetch configuration and/or operational data from the device. + + In case a device returns another status code meaning "no data was found", it needs + to be configured with this setting to make the NED fully operational. + + + - restconf model-discovery (default enabled) + + Configure the NED to auto probe for models supported by the device. This API call is part of + the RESTCONF specification, but is not supported by all devices. Note: this setting is + automatically configured when one of the pre-set RESTCONF profiles is used. + + enabled - Enabled. + + disabled - Disabled. + + + - restconf capability-discovery (default enabled) + + Configure the NED to auto probe for capabilities supported by the device. This API call is + part of the RESTCONF specification, but is not supported by all devices. Note: this setting + is automatically configured when one of the pre-set RESTCONF profiles is used. + + enabled - Enabled. + + disabled - Disabled. + + + - restconf protocol (default default) + + Configure the protocol to be used by the NED when applying config to the device. By default + the standard RESTCONF protocol is used. For devices supporting the newer YANG-PACH extension + it is recommended to use "yang-patch" or "auto". The YANG-PATCH extension is superior to + standard RESTCONF since it does provide full transactionality when applying config to the device. + The setting "auto" does require that capability-discovery is enabled as well. + + default - Use standard RESTCONF. + + yang-patch - Use the YANG-PATCH extension. Only works with devices supporting YANG-PATCH. + + auto - Enable YANG-PATCH if device advertises support for it. Otherwise use default + RESTCONF. + + + - restconf model-download accept-header (default application/yang) + + Configure accept header to use by the built-in YANG downloader tool when fetching the models + from the device. + + +## 4.1. ned-settings accedian-skylight_rc restconf cache +-------------------------------------------------------- + + The NED is able to cache certain data that is typically probed for when a new connection is setup. + Caching has good impact on performance, since reduces the number of necessary round trips to the + device on fro subsequent connections. + + + - cache model (default disabled) + + Configure the NED to cache the list of models supported by the device. Using the cache in + combination with models discovery enabled does save one additional round trip to the device + upon each connect. + + enabled - Enabled. + + disabled - Disabled. + + + - cache capability (default disabled) + + Configure the NED to cache the list of capabilities supported by the device. Using the cache + in combination with capabilities discovery enabled does save one additional round trip to the + device upon each connect. + + enabled - Enabled. + + disabled - Disabled. + + + - cache url-base (default disabled) + + Configure the NED to cache the url base used by the device. Using the cache in combination + with url-base set to 'auto' does save one additional round trip to the device upon each + connect. + + enabled - Enabled. + + disabled - Disabled. + + +## 4.2. ned-settings accedian-skylight_rc restconf config +--------------------------------------------------------- + + Settings related to RESTCONF operations on config. + + + - config update-method (default patch) + + Configure NED behaviour when updating config on the device. + + patch - Update using merge. A RESTCONF PATCH call is used. + + put - Update using replace. A RESTCONF PUT call is used. + + + - config gather-updates-into-single-patch (default false) + + When set to true the NED tries to gather updates on leafs with the same parent into one single + PATCH call. When set to false the NED generates one PATCH for each update. Default: false. + + + - config force-top-node-prefix on-create (default true) + + On create operations. + + + - config force-top-node-prefix on-update (default false) + + On update operations (PATCH / PUT). + + + - config yang-patch update-method (default merge) + + Configure NED behaviour when updating config on the device. + + merge - Update using YANG-PATCH merge. + + replace - Update using YANG-PATCH replace. + + + - config get-method (default default) + + Configure NED behaviour when fetching config from the device when doing sync-from etc. + + default - A full depth RESTCONF GET call is issued on each top node in the + config tree. + + use-custom-get-callpoints - Configure custom call points in the schema model. These will used + as paths when reading operational data. See chapter 'Configuring + Custom Call Points' for more information. + + + - config device-requires-consecutive-gets (default true) + + A device with custom call points might require the NED to execute additional consecutive GET + calls on sub levels to fetch all data. Then configure this setting to true. Note: this setting + is automatically configured when one of the pre-set RESTCONF profiles is used. + + + - config custom-get-call-points + + Specify schema paths to be used as call points when the NED is doing RESTCONF GET calls. See + chapter 'Configuring Custom Call Points' for more information. Note: this setting is + automatically configured when one of the pre-set RESTCONF profiles is used. + + - path + + - query depth + + Used to limit the number of levels of child nodes returned by the server. + + - query fields + + Used to identify data nodes within the target resource to be retrieved (see RFC8040 for + format details). + + - list-entry query depth + + Used to limit the number of levels of child nodes returned by the server. + + - list-entry query fields + + Used to identify data nodes within the target resource to be retrieved (see RFC8040 for + format details). + + - sub-nodes + + Use this call point to populate one of its sub nodes. When a request for the path that + corresponds to the sub node, the NED will use this call point instead, together with a query + composed by the path as a fields query and optionally a depth query configured for this + entry. + + - fields + + + - config append-content-config-query (default false) + + Appends the content=config query to the url on all GET calls. This instructs the device to + filter out operational data from the dumps to be returned. This can have good impact on + sync-from performance. Required that the content query feature is supported by the device. + + +### 4.2.1. ned-settings accedian-skylight_rc restconf config deviations +----------------------------------------------------------------------- + + Configure NED adaptions for device deviations. + + + - deviations list-entry move method (default default) + + The RESTCONF protocol does specify special REST operations to use when moving or inserting + entries in lists that are ordered by user (a certain YANG annotation). Some devices + can not handle move operations properly. This does apply to older versions of ConfD. + For such devices the NED can be configured to implement a move by doing a delete on the entry + followed by a insert on the right position. + + delete-and-insert - delete-and-insert. + + default - default. + + + - deviations list-entry update wrap-in-list (default true) + + When updating configuration inside a list entry, the NED by default wraps the payload in a list. + + Example: + Updating the leaf X inside the entry with key KEY=100 in the list LIST. + The HTTP PATCH is used: + + RESTCONF PATCH :: /restconf/data/LIST=100 + { + "LIST":[{ + "KEY":"100", + "X":"FOO" + }] + } + + Some devices require the payload to point at the entry itself. Example: + + RESTCONF PATCH :: /restconf/data/LIST=100 + { + "LIST":{ + "KEY":"100", + "X":"FOO" + } + } + + This NED setting makes the NED adapt accordingly. + + +## 4.3. ned-settings accedian-skylight_rc restconf live-status +-------------------------------------------------------------- + + NED settings related to RESTCONF operations for operational data. + + + - live-status get-method (default nearest-container) + + Configure NED behaviour when fetching operational data from the device. + + nearest-container - Execute a RESTCONF GET using a path representing nearest + container / list entry in the requested path. + + top-nodes - Execute a RESTCONF GET using a path representing the top node of + the requested path. + + use-custom-get-callpoints - Configure custom call points in the schema model. These will be + used as paths when reading operational data. + + + - live-status append-content-nonconfig-query (default false) + + Appends the content=nonconfig query to the url on all live-status GET calls. This instructs + the device to filter out config data from the dumps to be returned. Required that the content + query feature is supported by the device. + + + - live-status device-requires-consecutive-gets (default true) + + A device with custom call points might require the NED to execute additional consecutive GET + calls on sub levels to fetch all data. Then configure this setting to true. Note: this setting + is automatically configured when one of the pre-set RESTCONF profiles is used. + + + - live-status custom-get-call-points + + Specify schema paths to be used as call points when the NED is doing RESTCONF GET calls. See + chapter 'Configuring Custom Call Points' for more information. Note: this setting is + automatically configured when one of the pre-set RESTCONF profiles is used. + + - path + + - query depth + + Used to limit the number of levels of child nodes returned by the server. + + - query fields + + Used to identify data nodes within the target resource to be retrieved (see RFC8040 for + format details). + + - list-entry query depth + + Used to limit the number of levels of child nodes returned by the server. + + - list-entry query fields + + Used to identify data nodes within the target resource to be retrieved (see RFC8040 for + format details). + + - sub-nodes + + Use this call point to populate one of its sub nodes. When a request for the path that + corresponds to the sub node, the NED will use this call point instead, together with a query + composed by the path as a fields query and optionally a depth query configured for this + entry. + + - fields + + +## 4.4. ned-settings accedian-skylight_rc restconf notif +-------------------------------------------------------- + + Configure notification streams available on the device. + + + - notif inactive-stream-reset timeout (default 0) + + Configure the maximum allowed number of seconds of inactivity on a stream. The value 0 means + indefinite time. + + + - notif automatic-stream-discovery (default enabled) + + Let the NED automatically probe the device for supported streams. + + enabled - Enabled. + + disabled - Disabled. + + + - notif preferred-encoding (default xml) + + json - JSON encoding. + + xml - XML encoding. + + + - notif stream + + Manually configure info about stream on the device. This is useful when interacting with + devices not capable of advertising the supported streams automatically. + + - name + + Name of the stream. + + - path + + The path to access the stream. + + - replay-support (default false) + + Replay support. Set to true if device supports it. + + - description + + Description of this stream. + + +## 4.5. ned-settings accedian-skylight_rc restconf yang-push +------------------------------------------------------------ + + Experimental feature. Configure yang-push enabled streams / datastores available on the device. + + - restconf yang-push + + - name + + Name of emulated YANG-PUSH telemetry stream. + + - subscription-type + + Spceify type of subscription for this emulated stream. + + periodic - periodic. + + on-change - on-change. + + - xpath-filter + + Specify the xpath to the location in the device datatree to subscribe events from. + + - encoding (default xml) + + Specify the encoding type to be used for the telemetry data delivered as a notification to + NSO. + + json - json. + + xml - xml. + + + Either of: + + - devices device ned-settings accedian-skylight_rc restconf yang-push device datastore + + OR: + + - devices device ned-settings accedian-skylight_rc restconf yang-push device stream + + - on-change-settings dampening-period (default 0) + + Specify dampening period a dampening period to be used to specify the interval that has to + pass before successive update records for the same subscription are generated. + + - on-change-settings sync-on-start (default true) + + When set to 'true' (default), in order to facilitate a receiver's synchronization, a full + update is sent, via a 'push-update' notification, when the subscription starts. + + - on-change-settings excluded-change + + Used to restrict which changes trigger an update. For example, if a 'replace' operation is + excluded, only thecreation and deletion of objects are reported. + + create - A change that refers to the creation of a new + datastore node. + + delete - A change that refers to the deletion of a + datastore node. + + insert - A change that refers to the insertion of a new + user-ordered datastore + node. + + move - A change that refers to a reordering of the target + datastore node. + + replace - A change that refers to a replacement of the target + datastore node's + value. + + - periodic-settings period + + Specify the periodicity of the telemetry updates sent on this subscription. + + - periodic-settings anchor-time + + Designates a timestamp before or after which a series of periodic push updates are + determined. The next update will take place at a point in time that is a multiple of a + period from the 'anchor-time'. + + +# 5. ned-settings accedian-skylight_rc logger +--------------------------------------------- + + Settings for controlling logs generated. + + + - logger level (default info) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - logger java (default true) + + Toggle logs to be added to ncs-java-vm.log. + + +# 6. ned-settings accedian-skylight_rc general +---------------------------------------------- + + General NED settings. + + +## 6.1. ned-settings accedian-skylight_rc general capabilities +-------------------------------------------------------------- + + Settings related to device capabilities. + + + - capabilities strict-model-revision-check (default true) + + Configure the NED to do a strict revision check of the models published if possible. With this setting + enabled the exact revision needs to match the corresponding model built into the NED. Otherwise support + for it will be dropped by NSO. I.e not possible to read or write config using that model. + + + - capabilities defaults-mode-override + + Configure default value mode. + + report-all - Default mode 'report-all'. + + explicit - Default mode 'explicit'. + + trim - Default mode 'trim'. + + + - capabilities regex-exclude + + Configure a pattern for matching models to exclude from the capabilities list advertised by the device. + To be used to limit the scope of models registered into NSO by the NED. + + - pattern + + + - capabilities regex-include + + Configure a pattern for matching models to include from the capabilities list advertised by the device. + To be used to limit the scope of models registered into NSO by the NED. + + - pattern + + + - capabilities inject + + Configure additional names of models / urn:s to include in the capabilities list. If a device + is not able to advertise any capability list, the names of the models to be used must be + manually added to this inject list. + + - capa + + +## 6.2. ned-settings accedian-skylight_rc general config +-------------------------------------------------------- + + General settings related to config handling. + + + - config trans-id-method (default disabled) + + A transaction id is a hash that the NED optionally can calculate upon operations like commit and + check-sync. This NED does by default have trans-id calculation disabled. + If the NED is connected to a RESTCONF device that supports the "Last-Modified" time stamp header it can + use this feature to calculate a transaction id. This is a fast trans-id method. + + If the NED is connected to a RESTCONF device that supports the "Etag" header it can use this feature to + calculate a transaction id. This is also a fast trans-id method. + + If the NED is connected to a RESTCONF device that supports the "content=config" query, the config-hash + method can be used instead. This method does however require a full fetch of config. I.e it is much + slower than the time stamp and etag methods. + + last-modified-timestamp - Use the 'Last-Modified' http header in the response from a RESTCONF + GET call. Use this setting only with devices that supports it. + + etag - Use the 'Etag' http header in the response from a RESTCONF GET + call. Use this setting only with devices that supports it. + + config-hash - Calculate a transaction id based on the config dumps received from + the device. + + disabled - Disable the calculation of transaction id completely. + + + - config inbound-transforms + + Configure the following built-in transforms to be applied on the inbound payload before it is + passed to NSO. + + sort-keys - sort-keys. + + trim-namespace - trim-namespace. + + restore-namespace - restore-namespace. + + restore-identityrefs - restore-identityrefs. + + + - config filter-unmodeled (default false) + + Filter all nodes that are not represented in the YANG schema from the JSON payload received + from the device, before passing it to NSO. This can be useful if config applied to the device + is not displayed properly in NSO. Some versions of NSO have problems reading JSON payloads + containing unmodelled data. + + + - config filter-invalid-list-entries (default false) + + Filter all config data list entry nodes containing incomplete key sets. A list entry that does + not contain complete key sets will make NSO bail out the read operation completely. This + setting will prevent such issues. + + + - config partial-sync-from do-full-sync-from-on-error (default true) + + If a partial-sync-from operation fails, the NED can automatically try a full sync-from instead. This is + the default behaviour. The main reason is that the partial show feature is used internally by NSO during + abort. I.e when a commit has failed and NSO tries to calculate a reverse diff for restoring the device + to its original state. In this case it is better to let the NED revert to a full sync-from instead of + bailing out. The latter would result in a device in unknown state. Set this setting to false to instead + let the NED bail out on error. + + +## 6.3. ned-settings accedian-skylight_rc general live-status +------------------------------------------------------------- + + General settings related to live-status. + + + - live-status show-stats-filter (default false) + + Use the filter API from NSO to do live-status requests. This API is more flexible and will + result in fewer round-trips to the device and possibly less data transferred from the device. + The live-status time-to-live settings are not applicable when using this API. + + + - live-status restore-namespaces-and-identityrefs (default true) + + Make the NED go through the operational data dump and fix/restore the namespace information + and prefixes on identityref values etc. This is a common problem on many Skylight devices. NSO + is very strict about namespaces and will typically bail out the operation completely if it + finds a namespace issue. This setting will prevent such issues. Enabled by default. + + + - live-status filter-unmodeled (default false) + + Filter all nodes that are not represented in the YANG schema from the JSON payload received + from the device, before passing it to NSO. This can be useful if config applied to the device + is not displayed properly in NSO. Some versions of NSO have problems reading JSON payloads + containing unmodelled data. + + + - live-status inbound-transforms (default sort-keys) + + Configure the following built-in transforms to be applied on the inbound payload before it is + passed to NSO. + + sort-keys - sort-keys. + + trim-namespace - trim-namespace. + + restore-namespace - restore-namespace. + + restore-identityrefs - restore-identityrefs. + + + - live-status filter-invalid-list-entries (default false) + + Filter all config data list entry nodes containing incomplete key sets. A list entry that does + not contain complete key sets will make NSO bail out the read operation completely. This + setting will prevent such issues. + + +## 6.4. ned-settings accedian-skylight_rc general notif +------------------------------------------------------- + + General settings related to notifications. + + + - notif filter-unmodeled (default false) + + Filter all nodes that are not represented in the YANG schema from the JSON payload received + from the device, before passing it to NSO. This can be useful if config applied to the device + is not displayed properly in NSO. Some versions of NSO have problems reading JSON payloads + containing unmodelled data. + + + - notif inbound-transforms (default sort-keys) + + Configure the following built-in transforms to be applied on the inbound payload before it is + passed to NSO. + + sort-keys - sort-keys. + + trim-namespace - trim-namespace. + + restore-namespace - restore-namespace. + + restore-identityrefs - restore-identityrefs. + + + - notif filter-invalid-list-entries (default false) + + Filter all config data list entry nodes containing incomplete key sets. A list entry that does + not contain complete key sets will make NSO bail out the read operation completely. This + setting will prevent such issues. + + + - notif restore-namespaces-and-identityrefs (default true) + + Make the NED go through the operational data dump and fix/restore the namespace information + and prefixes on identityref values etc. This is a common problem on many Skylight devices. NSO + is very strict about namespaces and will typically bail out the operation completely if it + finds a namespace issue. This setting will prevent such issues. Enabled by default. + + diff --git a/accedian-skylight_rc/README-rebuild.md b/accedian-skylight_rc/README-rebuild.md new file mode 100644 index 00000000..c65e1d12 --- /dev/null +++ b/accedian-skylight_rc/README-rebuild.md @@ -0,0 +1,787 @@ +# Table of contents +------------------- +``` + 1. General + 1.1 Overview + 1.2 Using the built-in tool for YANG download + 1.3 Rebuild the NED with the downloaded YANG models + 1.4 Reload the NED package in NSO + 2. Removing all downloaded YANG models + 3. Using the built-in downloader tool for a custom download + 3.1 Creating a custom download + 4. Alternative download methods + 4.1 Copy the files into the NED source directory + 5. Rebuilding the NED using a custom NED-ID + 5.1 Rebuild with a custom NED-ID + 5.2 Exporting the rebuilt NED package + 6. YANG fixes applied when rebuilding the NED + 6.1 General + 6.2 Fixes listed by schema paths + 6.3 Fixes listed by YANG file paths + 6.4 Other fixes + 7. Advanced: repairing YANG modules + 7.1 Different kinds of YANG related issues + 7.2 Compile-time issues + 7.3 Run-time issues +``` + + +# 1. General +------------ + +## 1.1 Overview + +The accedian-skylight_rc NED is delivered without any YANG models in the package. + +The models do need to be downloaded, followed by a rebuild and reload of the package before the NED can be fully operational. + +This NED contains an optional built-in tool that makes downloading of the device models easy. + +Alternatively, manual download is possible as described in chapter 4. + +The NED package (i.e with no device models) must first be properly configured in a running NSO environment prior to usage of the downloader tool. The steps described in chapter 1.1 through 1.3 in the README.md must be done prior to this step. + +## 1.2 Using the built-in tool for simple YANG download + +The downloader tool is implemented as an NSO RPC which can be invoked for instance from the NSO CLI. + +When the tool is executed the NED will automatically connect to the device for which the YANG models shall be downloaded. The device will be requested to return a list of supported models when the NED is probing it for capabilities. + +This list of models will be the base used by the tool when downloading the models. During operation the tool will also scan each downloaded YANG file for additional dependencies through import/include found. The tool will try to download all such dependency YANG files as well. + + Since currently all Accedian-skylight devices tested with this NED do not support direct YANG model download from the device, the models need to be fetched from elsewhere. + + By default the Accedian-skylight repository on github is used as source for all YANG files. + +### Simple Usage + + The Accedian-skylight device does not support model discovery as specified by RFC8040. + In order to download the yang models for the device, several configuration needs to be done on the NED side to be able to pull the model from the GIT repo. + + 1. Disabling auto-discovery features of the NED: + By default, NED will try to discover the RESTCONF URL base, along with capabilities and yang models. Since the device does not support these features, they need to be disabled: + ``` + ned-settings accedian-skylight_rc restconf url-base "/restconf" + ned-settings accedian-skylight_rc restconf model-discovery disabled + ned-settings accedian-skylight_rc restconf capability-discovery disabled + + ``` + Commit the changes and proceed to the next step. + + 2. Inject modules that needs to be downloaded. + By default, the NED has no knowledge of the yang module names that are needed, since the module names are not advertised by Accedian Skylight gateway. + In order to make the module names known to the NED, they have to be manually injected using ned-settings: + ``` + ned-settings accedian-skylight_rc general capabilities inject Accedian-alert + ned-settings accedian-skylight_rc general capabilities inject Accedian-alert-metric + ned-settings accedian-skylight_rc general capabilities inject Accedian-extensions + ned-settings accedian-skylight_rc general capabilities inject Accedian-metadata + ned-settings accedian-skylight_rc general capabilities inject Accedian-net-type + ned-settings accedian-skylight_rc general capabilities inject Accedian-sat + ned-settings accedian-skylight_rc general capabilities inject Accedian-sat-capabilities + ned-settings accedian-skylight_rc general capabilities inject Accedian-service + ned-settings accedian-skylight_rc general capabilities inject Accedian-service-endpoint + ned-settings accedian-skylight_rc general capabilities inject Accedian-service-endpoint-ne + ned-settings accedian-skylight_rc general capabilities inject Accedian-service-endpoint-nid + ned-settings accedian-skylight_rc general capabilities inject Accedian-service-endpoint-ssc + ned-settings accedian-skylight_rc general capabilities inject Accedian-service-endpoint-unmanaged + ned-settings accedian-skylight_rc general capabilities inject Accedian-session-rfc2544 + ned-settings accedian-skylight_rc general capabilities inject Accedian-session-throughput + ned-settings accedian-skylight_rc general capabilities inject Accedian-session-twamp-light + ned-settings accedian-skylight_rc general capabilities inject Accedian-session-y1564 + ned-settings accedian-skylight_rc general capabilities inject Accedian-throughput-capabilities + ned-settings accedian-skylight_rc general capabilities inject Accedian-twamp-light-capabilities + ``` + Note: this is the extensive list of modules. It is not mandatory to use all the modules - eg if Accedian-session-y1564 is not actually needed, it can be ommited in the list, thus NED will skip it for download. Once done, commit the ned-settings. + + 3. List the download profiles + To ease the download process, the NED implements necessary information to download the Accedian Skylight yang model from the following GIT repository: https://github.com/Accedian/skylight-yang.git + To list the download profile, run the following RPC: + ``` + admin@ncs# devices device rpc rpc-list-profiles list-profiles + profile { + name Accedian-skylight-git + description Download the Accedian-skylight models. Download is done from the Accedian Skylight github repo. + } + admin@ncs# + ``` + NOTE: the download profile is based on the 'release/24.09' tag. If another GIT tag is needed, please check chapter 3 for additional options + + 4. Downloading the models: + ``` + admin@ncs# devices device accsky-1 rpc rpc-get-modules get-modules profile Accedian-skylight-git + result + Fetching modules: + Accedian-alert - http://accedian.com/ns/yang/alert (4387 bytes) + fetching imported module Accedian-alert-type + skipping import 'ietf-yang-types', matches '(ietf-.*)' + Accedian-alert-metric - http://accedian.com/ns/yang/alert/metric (6328 bytes) + Accedian-extensions - http://accedian.com/ns/yang/extensions (1064 bytes) + Accedian-metadata - http://accedian.com/ns/yang/metadata (1700 bytes) + Accedian-net-type - http://accedian.com/ns/yang/types/net (3294 bytes) + skipping import 'ietf-inet-types', matches '(ietf-.*)' + Accedian-sat - http://accedian.com/ns/yang/session/sat (2531 bytes) + Accedian-sat-capabilities - http://accedian.com/ns/yang/session/cap/sat (2374 bytes) + fetching imported module Accedian-service-endpoint + fetching imported module Accedian-session-type + Accedian-service - http://accedian.com/ns/yang/service (6835 bytes) + fetching imported module Accedian-session + fetching imported module Accedian-service-endpoint-type + fetching imported module Accedian-service-type + fetching imported module Accedian-type + Accedian-service-endpoint - http://accedian.com/ns/yang/service/endpoint (6369 bytes) + Accedian-service-endpoint-ne - http://accedian.com/ns/yang/service/endpoint/ne (3053 bytes) + skipping import 'ietf-inet-types', matches '(ietf-.*)' + Accedian-service-endpoint-nid - http://accedian.com/ns/yang/service/endpoint/nid (2368 bytes) + Accedian-service-endpoint-ssc - http://accedian.com/ns/yang/service/endpoint/ssc (1707 bytes) + Accedian-service-endpoint-unmanaged - http://accedian.com/ns/yang/service/endpoint/unmanaged (1308 bytes) + Accedian-session-rfc2544 - http://accedian.com/ns/yang/session/sat/rfc2544 (13283 bytes) + skipping import 'ietf-inet-types', matches '(ietf-.*)' + skipping import 'ietf-yang-types', matches '(ietf-.*)' + Accedian-session-throughput - http://accedian.com/ns/yang/session/throughput (5999 bytes) + skipping import 'ietf-inet-types', matches '(ietf-.*)' + Accedian-session-twamp-light - http://accedian.com/ns/yang/session/twamp/light (8493 bytes) + skipping import 'ietf-inet-types', matches '(ietf-.*)' + Accedian-session-y1564 - http://accedian.com/ns/yang/session/sat/y1564 (14384 bytes) + skipping import 'ietf-inet-types', matches '(ietf-.*)' + skipping import 'ietf-yang-types', matches '(ietf-.*)' + Accedian-throughput-capabilities - http://accedian.com/ns/yang/session/cap/throughput (2692 bytes) + Accedian-twamp-light-capabilities - http://accedian.com/ns/yang/session/cap/twamp/light (2710 bytes) + Accedian-alert-type - http://accedian.com/ns/yang/alert/type (1366 bytes) + Accedian-session-type - http://accedian.com/ns/yang/session/type (3153 bytes) + Accedian-session - http://accedian.com/ns/yang/session (6067 bytes) + Accedian-service-endpoint-type - http://accedian.com/ns/yang/service/endpoint/type (3496 bytes) + Accedian-service-type - http://accedian.com/ns/yang/service/type (823 bytes) + Accedian-type - http://accedian.com/ns/yang/types (2199 bytes) + fetched and saved 25 yang module(s) to /Users/lpanduru/01_WORK/03_STASH/accedian-skylight_rc/test/drned/drned-ncs/packages/accedian-skylight_rc/src/yang + admin@ncs# + ``` + After this step, the NED contains the Accedian Skylight yang models in the proper location, which means the NED needs to be rebuilt with the new model. + +For more advanced options using the downloader tool, see chapter 3. + +Chapter 5('rpc get-modules') in README.md describes more details about the downloader tool, including all available command line arguments. + +## 1.3 Rebuild the NED with the downloaded YANG models +The NED must be rebuilt when the device models have been downloaded and stored properly. + +Very often there are known issues related to building the device YANG models. Such issues +typically cause compiler errors or unwanted runtime errors/behaviours in NSO. + +The NED is configured to take care of all currently known build/runtime issues. It will automatically +patch the problematic files such that they do build properly for NSO. This is done using a set of YANG +build recipes bundled with the NED package. + +Note that the work with adapting the YANG build recipes is an ongoing process. If new issues are found +the Cisco NSO NED team will update the recipes accordingly and then release a new version of the NED. + +It is strongly recommended that end users report new YANG build issues found back to the Cisco NSO NED team +through a support request + +The NED provides two alternatives for rebuilding. It can be done either through NSO using a built-in tool, or by invoking gnu make in an external shell. + +### Rebuild using the built-in tool + +The built-in rpc *rebuild-package* does rebuild the NED package by automatically invoking gnu make in the source directory of the package installation root. + +``` +admin@ncs# devices device dev-1 rpc rpc-rebuild-package rebuild-package +``` + +Note: this rpc can take a long time to finish. This is because compiling YANG models is a very time consuming task. + +Additional arguments: + +``` +verbose : Print the full output returned from gnu make. By default the output is only printed upon errors. +profile : Apply a certain build profile when rebuilding. +ned-id : Parameters relevant for customizing the NED ID. See chapter 5 for further info. +``` + +### Rebuild using a separate shell + +The NED must be rebuilt from inside the NED package installation root (i.e $NED_ROOT_DIR configured in chapter 1.1 in README.md). + +To rebuild the NED do as follows: + +``` +> cd $NED_ROOT_DIR/src +> make clean all +``` + + +## 1.4 Reload the NED package in NSO +When the NED has been successfully rebuilt with the device models it is necessary to reload the package +into NSO. + +### Reloading + +Use the following NSO CLI command: + +``` +admin@ncs# packages reload +``` + +Note, if the NED packages has been rebuilt with a new NED-ID as described in chapter 5 it will +be necessary to add 'force' to the reload command: + +``` +admin@ncs# packages reload force +``` + + + +# 2. Removing all downloaded YANG models + +If a new set of YANG models is going to be downloaded for an already installed and reloaded NED it is highly recommended to first remove all old device YANG models from the previous download. + + +## 2.1 Clean the NED source directory + +Use the following make target to remove all previously downloaded YANG. + +### Clean using the built-in tool + +``` +admin@ncs# devices device dev-1 rpc rpc-clean-package clean-package +``` + +### Clean using a separate shell + +``` +> cd $NED_ROOT_DIR/src +> make distclean +``` + + + +# 3. Using the built-in downloader tool for a custom download + +Using a pre-configured download profile is the easiest way for downloading the device YANG models. + +In case this is not a suitable approach, the downloader tool has a number of additional arguments that allows +for doing a custom download. For instance by limiting the scope for downloading the YANG models. + +## 3.1 Creating a custom download + +Use the 'module-include-regex' and 'module-exclude-regex' to customize the download scope. +Both arguments shall be specified as regular expressions. + +The easiest way to get the arguments right is by using the RPC named 'list-modules' + + Note: the Accedian Skylight does not support model discovery for listing modules. For this reason, the modules needs to be specified using ned-settings. Please check chapter 1.2 to see how to inject modules. Once the models are known to the NED, custom download procedure can be follwed. + By default, the NED implements a download profile that is based on the Accedian Skylight yang model git repo, for the tag release/24.09. + If another git tag is needed, a custom download RPC can be used from the NED side: + + ``` + admin@ncs# devices device accsky-1 rpc rpc-get-modules get-modules remote { git { repository https://github.com/Accedian/skylight-yang.git checkout origin/release/24.09 dir skylight-gateway/accedian/public } } module-exclude-regex "(ietf-.*)" + ``` + + Explanations: + - remote: this is the method of download. in the above example, we are using git method + - repository: this is the git repository + - checkout: this is the git tag or branch to be used for the model + - dir: the path in the git repo where the models are stored + - module-exclude-regex: regular expression for modules to be excluded from the download. in the above example, all modules matching ietf-.* are excluded because they are already included in NSO distribution + +See chapter 5 in README.md for more details about the RPC commands get-modules and list-modules. + + + +# 4. Alternative download methods + +The device models can of course also be downloaded manually. To use this option the NED package +must first be unpacked. The steps described in README.md chapter 1.1 must be done first. It is preferred to do +the chapter 1.2 and 1.3 steps in README.md as well. + + +## 4.1 Copy the files into the NED source directory + +When the YANG models have been downloaded manually, all the files need to be copied into the source +directory of the NED installation: + +``` +> cp /*.yang $NED_ROOT_DIR/src/yang +``` + +Note, YANG files with names like `@.yang` need to be renamed to .yang. +This is because of a limitation in the current NED make system. + +This manual procedure is equivalent to using the downloader tool with a path to local directory +as remote source. See chapter 5('rpc get-modules') in README.md for more info. + +Please do not remove any of the the yang files used internally by the NED in the $NED_ROOT_DIR/src/yang + +This applies to the following files: + +``` +tailf-internal-rpcs.yang +tailf-internal-rpcs-custom.yang +tailf-ned-accedian-skylight_rc-meta.yang +tailf-ned-accedian-skylight_rc-meta-custom.yang +tailf-ned-accedian-skylight_rc-oper.yang +tailf-ned-accedian-skylight_rc-stats.yang +``` + +The best way to clean the source directory is to follow the steps in chapter 5. + +# 5. Rebuilding the NED using a custom NED-ID + +A common use case is to have many different versions of the same device in the network controlled +by NSO. Each device will then have its unique set of YANG files which this NED has to be rebuilt for. + +To setup NSO for this kind of scenario, each built flavour of the built NED must have its own unique NED-ID. + +This will make NSO allow multiple versions of the same NED package to co-exist. + +Rebuilding with a custom NED-ID can be done in the alternative ways: + +1. Through the built-in rpc, using the following additional arguments: + - suffix + - major + - minor +2. Through gmake in a separate shell, using the following additional make variables: + - NED_ID_SUFFIX + - NED_ID_MAJOR + - NED_ID_MINOR + +The default NED-ID is: accedian-skylight_rc-gen-3.0 + + +## 5.1 Rebuild with a custom NED-ID + +Do as follows to build each flavour of the accedian-skylight_rc NED. Do it in iterations, one at the time: + + 1. Follow the instructions in chapter 1.1 to 1.3 in README.md to unpack the NED and install a device instance + using it. Make sure a unique location is selected and update the environment variable $NED_ROOT_DIR + accordingly and configure a device instance. + + 2. Follow the instructions in chapter 1.2 to download the YANG models matching the configured + device. + + 3. Follow the instructions in chapter 1.3 to rebuild the NED. + + #### Alternative 1: Use the built-in rpc to rebuild with custom NED-ID + + Use any combination of the additional *ned-id* arguments *major, minor* and *suffix* + + Two examples showing NED-ID adapted for "device version" 21.6: + + ``` + admin@ncs# devices device dev-1 rpc rpc-rebuild-package rebuild-package ned-id suffix -21.6 + ``` + + This will generate a NED-ID like: 'accedian-skylight_rc-r21.6-gen-1.0' + + ``` + admin@ncs# devices device dev-1 rpc rpc-rebuild-package rebuild-package ned-id major 21 minor 6 + ``` + + This will generate a NED-ID like: 'accedian-skylight_rc-gen-21.6' + + #### Alternative 2: Rebuild the NED using gnu make from a shell + + Add any combination of the additional make variables 'NED_ID_SUFFIX','NED_ID_MAJOR' and 'NED_ID_MINOR' to the command line. + + Two examples showing NED-ID adapted for "device version" 21.6: + + ``` + > make NED_ID_SUFFIX=-r21.6 clean all + ``` + + This will generate a NED-ID like: 'accedian-skylight_rc-r21.6-gen-1.0' + + ``` + > make NED_ID_MAJOR=21 NED_ID_MINOR=6 clean all + ``` + + This will generate a NED-ID like: 'accedian-skylight_rc-gen-21.6' + + 4. Follow the instructions in chapter 1.4 to reload the NED package. The rebuilt NED package + will now have the NED-ID: `accedian-skylight_rc-gen-.` + + This is detected by NSO and will have the following side effects: + + - The default NED-ID will no longer exist after the packages has been reloaded + in NSO. Hence, any devices configured with the default NED-ID can no longer exist either. + + - It is recommended to delete all device instances using the default NED-ID before + reloading the packages in NSO. + + - It is necessary to use 'packages reload force' when reloading the packages in NSO. + + 5. Reconfigure the device instance from step #1 in this list. Now use the new NED-ID + + 6. Verify functionality by executing a 'sync-from' on the configured device instance. + + + +## 5.2 Exporting the rebuilt NED package + +When the NED has been rebuilt with a new NED-ID, it typically needs to be exported as a separate NED package. This new package will then be a ready to use NED containing the raw as well as the customized and compiled versions of the third party YANG models. This package can then be loaded into a new NSO instance etc just like any other NED. + +The NED has a built-in tool to make the export procedure easy. It is implemented as an additional rpc which can be invoked through the NSO CLI or any other northbound NSO interface. + +It copies all relevant elements from the "source" directory of rebuilt NED into a tar.gz archive file. The top dir of the archive will be named in accordance with the generated NED-ID, i.e. accedian-skylight_rc-gen-.. + +Usage: + +``` +admin@ncs# devices device dev-1 rpc rpc-export-package export-package +result ok +exported to: /tmp/ncs-6.2.2-accedian-skylight_rc-1.0.2-customized.tar.gz +``` + +Additional arguments: + +``` +destination : Destination directory for the exported tar.gz archive file. Default: /tmp +suffix : Specify a suffix to be appended to the name of the archive file. Default: -customized +``` + + +# 6. YANG fixes applied when rebuilding the NED + +Third party YANG files often have errors that can cause problems during compilation and/or at runtime. + +Common examples of such are besides pure YANG bugs, also specific NSO incompatible YANG constructs or that the device does not obey the models properly. + +The NED build system is instrumented with a set of tools that is able to automatically apply various kinds of fixes to the third party YANG files. These fixes, referred to as YANG recipes, ensure that the YANG files will compile properly when the NED package is being rebuilt. This of course only refers to the YANG file versions that have so far been verified by the Cisco NED team. New build time issues can still occur if the NED is rebuilt with unverified YANG file versions. + +**It is encouraged to contact the Cisco NED team on any such issue by opening a support ticket**. See chapter 8 in the README.md for further information. + +Below is a description of the YANG recipes currently in use by the accedian-skylight_rc NED. + + +# 7. Advanced: repairing YANG modules + +------------------------------------- + +Many third party YANG models are plagued with issues preventing them from being compiled and/or used with NSO. They simply need to be repaired. + +This NED package has been verified to work with device models and YANG model revisions listed in the README.md. Any issues that were discovered +in the listed revisions have been fixed, and the fixes are already included in the package. + +However, there is always a risk that new problems related to the YANG models will emerge. For example, if the NED is used together with YANG models that have not been verified by the Cisco NED team. Another example could be if the NED is used in a new use case that has not previously been verified. + +If this happens, new YANG repair might be necessary. + +**It is encouraged to contact the Cisco NED team for help in any such matter.** + +Create a support case with a description of the issue and the use case. Follow the instructions in the README.md. + +There is of course also a do it yourself option. This is mainly for the advanced user though. Good knowledge about the YANG modelling language is required. The NED is bundled with a set of tools to help automating YANG repair. This chapter is intended to give an overview of how to use them. + + + +## 7.1 Different kinds of YANG related issues + +When preparing the YANG modules for use with NSO, we can classify the kind issues found in the YANG modules as either to be of compile-time, or run-time character. + +With compile-time we mean an issue which needs to be fixed to be able to at all compile and load the YANG into NSO, whereas a run-time issue is anything that needs to be addressed after the YANG is successfully compiled (e.g. invalid constraints which are not met by actual device). + + + +## 7.2 Compile-time issues + +--------------------------- + +The first step before using the YANG modules in NSO is to make sure they can be compiled properly. For convenience, most standard issues found in YANG are by default automatically patched when compiling the package, in which case the resulting files will be found in *src/tmp-yang* (NOTE: the original YANG-files in src/yang are not modified). + +This is controlled by the make variable AUTOPATCH_YANG_NED. Usually it is enabled already in the NED make file *$NED_ROOT_DIR/src/Makefile*, like below: + +``` +AUTOPATCH_YANG_NED ?= yes +``` + +The auto patcher feature does only fix the most standard issues. + +If the YANG compilation still fails, like in the example below, further actions are necessary: + +``` +> cd $NED_ROOT_DIR/src +> make clean all +augmented/openconfig-network-instance@2022-07-04.yang:2876: error: the node 'protocol' from module 'openconfig-network-instance' (in node 'config' in module 'openconfig-network-instance' from 'openconfig-network-instance') is not found +augmented/openconfig-rib-bgp-attributes.yang:2284: error: the node 'endpoint' from module 'openconfig-network-instance' (in node 'state' in module 'openconfig-network-instance' from 'openconfig-network-instance') is not found +augmented/openconfig-rib-bgp-attributes.yang:2309: error: the node 'instance-id' from module 'openconfig-network-instance' (in node 'state' in module 'openconfig-network-instance' from 'openconfig-network-instance') is not found +``` + + + +Always try to avoid modifying the original files in *src/yang*. Instead use the available tools to do the patching at compile-time, with the results ending up in *src/tmp-yang* where the make process always places the actual files currently in use. + +The NED is bundled with three different YANG pre-processor tools that can be used to patch the YANG files at compile-time: + +- schypp + +- jypp + +- ypp + + + +### 7.2.1 The schypp tool + +This is the schema aware YANG pre-processor tool. It is automatically invoked at compile-time and reads directives from the file *$NED_ROOT_DIR/src/customize-schema.schypp*. + +Each line in this file contains one directive, starting with one of the keywords add, remove, replace, or inline-type. This keyword is followed by the globally unique schema-path for the node to 'operate' on. No line breaks allowed. + +If prefix is needed for a non-unique name in the path, the prefix must be the same as declared by the module. Depending on the type of operation the schema-path shall be followed by '::' (i.e. two colons) followed by further arguments as needed. The details for each type is described below. + +The following directives are supported: + +- inline-type +- add +- delete +- replace + + + +#### The inline-type + +This is applicable for leafs of type leafref. The tool looks up the type of the referred leaf and inlines the same type on the referee. + +Very useful for all kinds of YANG issues related to leafrefs etc. + +Syntax: + +``` +inline-type +``` + +Example: + +``` +inline-type /oc-netinst:network-instances/network-instance/protocols/protocol/name +``` + + + +#### The add + +For adding YANG statements to the schema. + +Syntax: + +``` +add :: +``` + +Example: + +``` +add /configuration/protocols/l2circuit/neighbor::uses apply-advanced; +add /configuration/protocols/l2circuit/neighbor/interface::{ min-elements 1; max-elements 10; } +add /configuration/protocols/l2circuit/neighbor::@file-with-YANG-snippet +``` + + + +#### The remove + +For removing YANG statement from the schema. + +Syntax: + +``` +remove :: +``` + +Example: + +``` +remove /configuration/routing-instances/instance/protocols/vpls/mesh-group/neighbor/vpls-id-list::ordered-by +remove /oc-acl:acl/acl-sets/acl-set/acl-entries/acl-entry/actions/config/forwarding-action::mandatory +``` + + + +#### The replace + +For replacing YANG statements in the schema. In this case the matching statement (must be unique match) is replaced by the +given statement(s), same rules for stmt-match and YANG-stmt(s) as for +add/remove. + +``` +replace :::: +``` + + + +### 7.2.2 The jypp tool + +This tool is implemented in Java. Hence its name. Everything you can do with the schypp tool can also be done with this tool. The big difference is that it is YANG model aware instead of schema aware. It needs a "path" in the YANG model file to the node/area to operate on. + +To find out the path, do as follows: + +1. Find out the name of the file that contains the declaration of the node to operate on. +2. Open the file and search for the node to operate on. +3. Run the jypp tool from a shell like below: + +``` +> cd $NED_ROOT_DIR/src +> ./tools/jypp --print-path= tmp-yang/.yang +``` + +Example: + +The range for MPLS label values, specified in the file openconfig-mpls-types.yang line 278, needs to be modified. + +``` +typedef mpls-label { + type union { + type uint32 { + range 13..1048575; + } + .. + .. +``` + +Run the tool to find out the YANG path to it: + +``` +> ./tools/jypp --print-path=280 tmp-yang/openconfig-mpls-types.yang +openconfig-mpls-types.yang:280 /#mpls-label/type/#uint32/range +``` + +The path is: /#mpls-label/type/#uint32/range + + + +With a correct path following operations are supported by the jypp tool: + +- --add-stmt +- --remove-stmt +- -- remove-all +- --replace-stmt + +#### The --add-stmt + +Add extra statement to the node identified by the YANG path + +Syntax: + +``` +> ./tools/jypp --add-stmt=:: +``` + +Example: + +``` +> ./tools/jypp --add-stmt=/#mpls-label/type/#uint32::description "This is an addition"; tmp-yang/openconfig-mpls-types.yang +``` + +#### The --remove-stmt + +Remove a statement identified by the YANG path. + +Syntax: + +``` +> ./tools/jypp --remove-stmt= +``` + +Example: + +``` +> ./tools/jypp --remove-stmt=/#mpls-label/type/#uint32 tmp-yang/openconfig-mpls-types.yang +``` + +#### The --replace-stmt + +Replace a statement on the node identified by the YANG path. + +Syntax: + +``` +> ./tools/jypp --replace-stmt=:: +``` + +Example: + +``` +> ./tools/jypp --replace-stmt==/#mpls-label/type/#uint32/range::"range 13..1048575;" tmp-yang/openconfig-mpls-types.yang +``` + + + +Test executing the new jypp rules from a shell and verify that the corresponding files in *tmp-yang* have been modified accordingly. When you have a set of jypp rules that works, it is time to install them into the make machinery. This ensures that they are automatically applied every time the NED package is re-built. + +Open the file *$NED_ROOT_DIR/src/ned-custom.mk*. Scroll to the make target *do-profile-defaul*t and add the new jypp rules to it. + +Example: + +``` +do-profile-default: + ./tools/jypp --replace-stmt=/#mpls-label/type/#uint32/range::"range 13..1048575;" \ + 'tmp-yang/openconfig-mpls-types.yang' &> /dev/null || true +.PHONY: do-profile-default +``` + + + +### 7.2.3 The ypp tool + +This is the oldest of the three YANG pre-processor tools. It is a pattern match oriented text processing utility. The patterns are made of regular expressions. The ypp tool is basically like an advanced *sed*. Everything you can do with schypp and jypp can also be done with ypp. In most cases schypp and jypp are superior and more easy to use. However, sometimes the ypp tool is the only possible option. + +Syntax: + +``` +./tools/ypp --from= --to= +``` + +Example: + +``` +./tools/ypp --from="(list\s+route\s+[^;]+)(route-type)([^;]+;)" --to="\g<1>\g<3>" 'tmp-yang/srl-ip-route-tables.yang' +./tools/ypp --from="(leaf\s+(dot1p-policy)\s+{[^}]+)(type\s+leafref[^}]+})" --to="\g<1>type string;" 'tmp-yang/srl-qos.yang' +``` + +Test executing the new ypp rules from a shell and verify that the corresponding files in *tmp-yang* have been modified accordingly. When you have a set of ypp rules that works, it is time to install them into the make machinery. This ensures that they are automatically applied every time the NED package is re-built. + +Open the file *$NED_ROOT_DIR/src/ned-custom.mk*. Scroll to the make target *do-profile-defaul*t and add the new jypp rules to it. + +Example: + +``` +do-profile-default: + ./tools/ypp --from="(list\s+route\s+[^;]+)(route-type)([^;]+;)" --to="\g<1>\g<3>" 'tmp-yang/srl-ip-route-tables.yang' +.PHONY: do-profile-default +``` + + + +### 7.2.4 Verify the YANG repair + +There is a separate make target to only do the 'pre-processing' of the YANG modules, i.e. automatic patching and application of 'local' schema customizations, it is run like this: + +``` +> cd $NED_ROOT_DIR/src +> make prepare-yang +``` + +When run, one can immediately see the resulting YANG modules in the directory *src/tmp-yang* (i.e. this is what will be used by the NED at run-time). + +## 7.3 Run-time issues + +---------------------- + +After all YANG modules can be compiled and loaded into NSO, there might still be issues which are only discovered at run-time. These might be constraints that are present in the YANG module but which are not strictly met by the device, such as integer value ranges or leafref targets. These kind of issues need to be investigated with real config from the device, hence a NED instance needs to be able to connect to device and do a full sync-from. + +For example, if the device returns config containing a leaf of type leafref whose target is missing, when doing a full sync-from, NSO might say something like: + +``` +admin@ncs# devices device dev-1 sync-from +result false +info illegal reference devices device dev-1 config components component 0/0 subcomponents subcomponent 0/0-Virtual-Motherboard name +``` + +This indicates that the device is not fully compliant with the YANG models it is using. Hence, the YANG needs further repair. + +Use the same toolkit as described in 6.2 to fix run-time issues. + +The issue described above can for example be repaired with an additional schypp statement: + +``` +inline-type /components/component/subcomponents/subcomponent/name +``` diff --git a/accedian-skylight_rc/README.md b/accedian-skylight_rc/README.md new file mode 100644 index 00000000..02686616 --- /dev/null +++ b/accedian-skylight_rc/README.md @@ -0,0 +1,1335 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in RPC actions + 5.1. rpc clean-package + 5.2. rpc compile-modules + 5.3. rpc export-package + 5.4. rpc get-modules + 5.5. rpc list-modules + 5.6. rpc list-profiles + 5.7. rpc patch-modules + 5.8. rpc rebuild-package + 5.9. rpc show-default-local-dir + 5.10. rpc show-loaded-schema + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. NED connection methods + 11. NED Notifications + ``` + + +# 1. General +------------ + + This document describes the accedian-skylight_rc NED. + + Accedian Skylight is a RESTCONF NED based on Accedian 3PY yang models. The 3PY yang models are slightly parsed while the NED is recompiled in order to make the model RESTCONF compliant. + + The NED is in POC state and it is not fully tested for full-RESTCONF CRUD operaitions. + + The NED connection with the device is based on either mTLS certificates or authorization bearer. + Please check additional content in this readme and README-ned-settings.md + + IMPORTANT: + This NED is delivered without any of the device YANG models bundled to the NED package. + + It is required to download the YANG files separately and rebuild the NED package before the NED is + fully operational. See the README-rebuild.md for further information. + + The recommended way to do this is by following the steps below: + + 1. Install and setup the NED. See chapter 1.1 to 1.3 + 2. Download the YANG models. See chapter 1.1 in README-rebuild.md for the recommended method (alternatives are described in + README-rebuild.md chapter 2 and 3). + 3. Rebuild the NED. See chapter 1.3 in README-rebuild.md (an alternative with a custom NED-ID is described in README-rebuild.md chapter 4). + 4. Reload the NED in NSO. See chapter 1.4 in README-rebuild.md + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + | | | + | README-rebuild.md | Detailed instructions on how to download the device YANG models and | + | | rebuilding the NED with them. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | Netsim device on port 7888, implements mTLS | + | | | | + | check-sync | no | Returns Unsupported | + | | | | + | partial-sync-from | no | Not supported | + | | | | + | live-status actions | no | Currently not needed | + | | | | + | live-status show | no | Does not support TTL'based data | + | | | | + | load-native-config | no | Not implemented | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + Custom NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | notifications | yes | Supports all Accedian Skylight notification streams (service, | + | | | session, service-endpoint) | + | | | | + | notification-stream- | yes | Supports automated stream discovery for notifs | + | autodiscovery | | | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Accedian Skylight Gateway | NA | NA | Accedian Skylight Gateway Rest | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + Verified YANG model bundles + ``` + +---------------------------+-----------------+------------------------------------------------------------+ + | Bundle | Version | Info | + +---------------------------+-----------------+------------------------------------------------------------+ + | Accedian | NA | Accedian Skylight 3PY yang model | + +---------------------------+-----------------+------------------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--accedian-skylight_rc-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-accedian-skylight_rc-1.0.1.signed.bin + > ./ncs-6.0-accedian-skylight_rc-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-accedian-skylight_rc-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-accedian-skylight_rc-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + **IMPORTANT**: + + This NED is delivered as an “empty” package, i.e without any device YANG models bundled. + It must be rebuilt with the device YANG models to become operational. + + The procedure to rebuild the empty NED (described in the README-rebuild.md) shall typically + be done in a lab environment. For this step a “local install” of the NED shall be used. + It is not suitable to use “system install” here since it is intended for production systems only. + + Once this NED has been rebuilt with the device YANG and exported to one or many + separate tar.gz customized NED packages, a “system installation” can be used on them. + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `accedian-skylight_rc-.`: + + ``` + > tar xfz ncs-6.0-accedian-skylight_rc-1.0.1.tar.gz + > ls -d */ + accedian-skylight_rc-gen-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package accedian-skylight_rc-gen-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package accedian-skylight_rc-gen-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-accedian-skylight_rc-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package accedian-skylight_rc-gen-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/accedian-skylight_rc-gen-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-accedian-skylight_rc-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-accedian-skylight_rc-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install accedian-skylight_rc-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-accedian-skylight_rc-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-accedian-skylight_rc-gen-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type generic ned-id accedian-skylight_rc-gen-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + **IMPORTANT**: + + The *device-type* shall always be set to *generic* when configuring a device instance + to use a 3PY NED. A common mistake is configuring it as *netconf*, which will cause + NSO to use its internal netconf client instead. + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-accedian-skylight_rc-gen-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings accedian-skylight_rc logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings accedian-skylight_rc logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.AccedianSkylight \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings accedian-skylight_rc logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings accedian-skylight_rc logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + The following example represents a common use-case of creating service-endpoints, sessions and services: + + ``` + admin@ncs(config-config)# + admin@ncs(config-config)# service-endpoints service-endpoint NED-SE-EP01 + admin@ncs(config-service-endpoint-NED-SE-EP01)# endpoint-name "NED_SE_EP01" + admin@ncs(config-service-endpoint-NED-SE-EP01)# description "NED Session sender EP01" + admin@ncs(config-service-endpoint-NED-SE-EP01)# type ne-endpoint + admin@ncs(config-service-endpoint-NED-SE-EP01)# config ne-config ne-id p.P-.881-KiKIKKKk. + admin@ncs(config-service-endpoint-NED-SE-EP01)# config ne-config vlan-id 11 + admin@ncs(config-service-endpoint-NED-SE-EP01)# config ne-config ip 22.11.22.11 + admin@ncs(config-service-endpoint-NED-SE-EP01)# exit + admin@ncs(config-config)# service-endpoints service-endpoint NED-SE-EP02 + admin@ncs(config-service-endpoint-NED-SE-EP02)# endpoint-name "NED_SE_EP02" + admin@ncs(config-service-endpoint-NED-SE-EP02)# description "NED Session sender EP02" + admin@ncs(config-service-endpoint-NED-SE-EP02)# type ne-endpoint + admin@ncs(config-service-endpoint-NED-SE-EP02)# config ne-config ne-id p.P-.881-KiKIKKKk. + admin@ncs(config-service-endpoint-NED-SE-EP02)# config ne-config vlan-id 22 + admin@ncs(config-service-endpoint-NED-SE-EP02)# config ne-config ip 11.22.11.22 + admin@ncs(config-service-endpoint-NED-SE-EP02)# exit + admin@ncs(config-config)# + admin@ncs(config-config)# sessions session NED_Test_Session_01 + admin@ncs(config-session-NED_Test_Session_01)# session-name NED_Test_Session_01 + admin@ncs(config-session-NED_Test_Session_01)# session-type twamp-light + admin@ncs(config-session-NED_Test_Session_01)# service-endpoints NED-SE-EP01 + admin@ncs(config-service-endpoints-NED-SE-EP01)# session-protocol twamp-light session-reflector admin-state true + admin@ncs(config-service-endpoints-NED-SE-EP01)# exit + admin@ncs(config-session-NED_Test_Session_01)# service-endpoints NED-SE-EP01 + admin@ncs(config-service-endpoints-NED-SE-EP01)# session-protocol twamp-light session-sender admin-state true + admin@ncs(config-service-endpoints-NED-SE-EP01)# session-protocol twamp-light session-sender reflector-udp-port 888 + admin@ncs(config-service-endpoints-NED-SE-EP01)# exit + admin@ncs(config-session-NED_Test_Session_01)# exit + admin@ncs(config-config)# sessions session NED_Test_Session_02 + admin@ncs(config-session-NED_Test_Session_02)# session-name NED_Test_Session_02 + admin@ncs(config-session-NED_Test_Session_02)# session-type twamp-light + admin@ncs(config-session-NED_Test_Session_02)# service-endpoints NED-SE-EP01 + admin@ncs(config-service-endpoints-NED-SE-EP01)# session-protocol twamp-light session-sender admin-state true + admin@ncs(config-service-endpoints-NED-SE-EP01)# session-protocol twamp-light session-sender reflector-ip 11.11.11.11 + admin@ncs(config-service-endpoints-NED-SE-EP01)# session-protocol twamp-light session-sender report-interval 35 + admin@ncs(config-service-endpoints-NED-SE-EP01)# session-protocol twamp-light session-sender test-packets payload-size 120 + admin@ncs(config-service-endpoints-NED-SE-EP01)# session-protocol twamp-light session-sender test-packets dscp 1 + admin@ncs(config-service-endpoints-NED-SE-EP01)# session-protocol twamp-light session-sender test-packets ttl 200 + admin@ncs(config-service-endpoints-NED-SE-EP01)# session-protocol twamp-light session-sender test-packets vlan-priority 7 + admin@ncs(config-service-endpoints-NED-SE-EP01)# exit + admin@ncs(config-session-NED_Test_Session_02)# service-endpoints NED-SE-EP02 + admin@ncs(config-service-endpoints-NED-SE-EP02)# session-protocol twamp-light session-sender admin-state true + admin@ncs(config-service-endpoints-NED-SE-EP02)# session-protocol twamp-light session-sender reflector-udp-port 888 + admin@ncs(config-service-endpoints-NED-SE-EP02)# exit + admin@ncs(config-session-NED_Test_Session_02)# exit + admin@ncs(config-config)# services service NED_service_1 + admin@ncs(config-service-NED_service_1)# service-name ned_service_1 + admin@ncs(config-service-NED_service_1)# sessions NED_Test_Session_01 + admin@ncs(config-sessions-NED_Test_Session_01)# exit + admin@ncs(config-service-NED_service_1)# sessions NED_Test_Session_02 + admin@ncs(config-sessions-NED_Test_Session_02)# exit + admin@ncs(config-service-NED_service_1)# exit + ``` + + Checking the NED device-native output: + + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name accsky-1 + data RESTCONF POST :: /restconf/data/Accedian-service-endpoint:service-endpoints + { + "Accedian-service-endpoint:service-endpoint":[{ + "endpoint-id":"NED-SE-EP01", + "endpoint-name":"NED_SE_EP01", + "description":"NED Session sender EP01", + "type":"Accedian-service-endpoint-type:ne-endpoint", + "config":{ + "Accedian-service-endpoint-ne:ne-config":{ + "ne-id":"p.P-.881-KiKIKKKk.", + "vlan-id":11, + "ip":"22.11.22.11" + } + } + }] + } + RESTCONF POST :: /restconf/data/Accedian-service-endpoint:service-endpoints + { + "Accedian-service-endpoint:service-endpoint":[{ + "endpoint-id":"NED-SE-EP02", + "endpoint-name":"NED_SE_EP02", + "description":"NED Session sender EP02", + "type":"Accedian-service-endpoint-type:ne-endpoint", + "config":{ + "Accedian-service-endpoint-ne:ne-config":{ + "ne-id":"p.P-.881-KiKIKKKk.", + "vlan-id":22, + "ip":"11.22.11.22" + } + } + }] + } + RESTCONF POST :: /restconf/data/Accedian-session:sessions + { + "Accedian-session:session":[{ + "session-id":"NED_Test_Session_01", + "session-name":"NED_Test_Session_01", + "session-type":"Accedian-session-type:twamp-light", + "service-endpoints":[{ + "endpoint-id":"NED-SE-EP01", + "session-protocol":{ + "Accedian-session-twamp-light:twamp-light":{ + "session-sender":{ + "admin-state":true, + "reflector-udp-port":888 + }, + "session-reflector":{ + "admin-state":true + } + } + } + }] + }] + } + RESTCONF POST :: /restconf/data/Accedian-session:sessions + { + "Accedian-session:session":[{ + "session-id":"NED_Test_Session_02", + "session-name":"NED_Test_Session_02", + "session-type":"Accedian-session-type:twamp-light", + "service-endpoints":[{ + "endpoint-id":"NED-SE-EP01", + "session-protocol":{ + "Accedian-session-twamp-light:twamp-light":{ + "session-sender":{ + "admin-state":true, + "reflector-ip":"11.11.11.11", + "report-interval":35, + "test-packets":{ + "payload-size":120, + "dscp":1, + "ttl":200, + "vlan-priority":7 + } + } + } + } + },{ + "endpoint-id":"NED-SE-EP02", + "session-protocol":{ + "Accedian-session-twamp-light:twamp-light":{ + "session-sender":{ + "admin-state":true, + "reflector-udp-port":888 + } + } + } + }] + }] + } + RESTCONF POST :: /restconf/data/Accedian-service:services + { + "Accedian-service:service":[{ + "service-id":"NED_service_1", + "service-name":"ned_service_1", + "sessions":[{ + "session-id":"NED_Test_Session_01" + },{ + "session-id":"NED_Test_Session_02" + }] + }] + } + } + } + admin@ncs(config-config)# + ``` + + Commiting the changes to the device: + + ``` + admin@ncs(config-config)#commit + Commit complete. + admin@ncs(config-config)# + ``` + + Checking if there are compare-config issues: (Note: compare-config output should be empty, otherwise something went wrong with the commit) + + + ``` + admin@ncs(config-config)# compare-config + admin@ncs(config-config)# + ``` + + Starting/Stopping sessions: + + ``` + admin@ncs(config-config)# sessions session NED_Test_Session_01 + admin@ncs(config-session-NED_Test_Session_01)# start + + ``` + + +# 5. Built in RPC actions +--------------------------------- + + ## 5.1. rpc clean-package + ------------------------- + + Cleans the NED package from all downloaded third party YANG files. + + Input arguments: + + - verbose + + Print the full clean output also for successful executions (otherwise only printed on errors). + + + ## 5.2. rpc compile-modules + --------------------------- + + Compile YANG modules, showing all non-fatal warnings found. + + Input arguments: + + - local-dir + + Path to the directory where the YANG files are found (defaults to src/yang in package). + + + - no-deviations + + Set to disable deviations. + + + ## 5.3. rpc export-package + -------------------------- + + Export the customized and rebuilt NED. The exported archive file can then be used to install the + NED package in other NSO instances. The name of the file will have the following format ncs----customized.tgz. + + Input arguments: + + - destination (default /tmp) + + Set destination directory for the exported archive file. + + + - suffix (default -customized) + + Configure a customized suffix to the name of the archive file. + + + ## 5.4. rpc get-modules + ----------------------- + + Fetch the YANG modules advertised by the device, from device or given source. + + Input arguments: + + - module-include-regex + + Regular expression matching all YANG models to be included in the download. Example: + 'openconfig-.*'. + + + - module-exclude-regex + + Regular expression matching all YANG models to be excluded from the download. Example: + 'tailf-.*'. + + + - namespace-include-regex + + Regular expression matching all namespaces to be included in the download. Example: + 'tailf-.*'. + + + - namespace-exclude-regex + + Regular expression matching all namespaces to be excluded from the download. Example: + 'tailf-.*'. + + + - module-additional-include-regex + + Regular expression matching additional YANG models that are not advertised by the device. For + instance vendor specific deviation models. Example: cisco-nx-openconfig-deviations.*. + + + - module-additional-exclude-regex + + Regular expression matching exceptions from the additional-include-regex. + + + - profile + + Use a download profile to match a predefined subset of matching YANG files. + + + - local-dir + + Path to the directory where the YANG files are to be copied (defaults to src/yang in package). + + + - ignore-errors + + Ignore errors during download. For example missing files of failed revision checks. + + + Either of: + + - remote device + + The device itself. + + OR: + + - remote dir + + A directory on the local host holding all YANG files. For instance a local clone of a git + repository. + + OR: + + - remote archive + + A path to a zip/tgz archive file containing the YANG files. + + OR: + + - remote git repository + + The URL to the git repository. Example: https://github.com/YangModels/yang.git. + + - remote git dir + + Path to a sub directory inside the git repo where the YANG files can be found. Example: + vendor/cisco/nx/10.1-2. + + - remote git checkout + + Optionally, a name of a branch/tag in the git repo where the YANG files can be found. + Example: master. + + - remote git include-dir + + Optional extra include paths to be used when searching for YANG files. Each include path + is relative to the git root directory. + + + ## 5.5. rpc list-modules + ------------------------ + + Use this command to list the YANG modules advertised by the device. Returns a list with module + names, including revision tag if available. + + Input arguments: + + - module-include-regex + + Regular expression matching all YANG models to be included in the download. Example: + 'openconfig-.*'. + + + - module-exclude-regex + + Regular expression matching all YANG models to be excluded from the download. Example: + 'tailf-.*'. + + + - namespace-include-regex + + Regular expression matching all namespaces to be included in the download. Example: + 'tailf-.*'. + + + - namespace-exclude-regex + + Regular expression matching all namespaces to be excluded from the download. Example: + 'tailf-.*'. + + + - module-additional-include-regex + + Regular expression matching additional YANG models that are not advertised by the device. For + instance vendor specific deviation models. Example: cisco-nx-openconfig-deviations.*. + + + - module-additional-exclude-regex + + Regular expression matching exceptions from the additional-include-regex. + + + - profile + + Use a download profile to match a predefined subset of matching YANG files. + + + ## 5.6. rpc list-profiles + ------------------------- + + List all predefined download profiles bundled with the NED. Including a short description of each. + + No input arguments + + + ## 5.7. rpc patch-modules + ------------------------- + + Patch YANG modules, to remove non-fatal warnings found. + + Input arguments: + + - local-dir + + Path to the directory where the YANG files are found (defaults to src/yang in package). + + + - no-deviations + + Set to disable deviations. + + + - output-dir + + Path to the directory where the patched YANG files are written (defaults to src/yang in + package), existing files will be renamed to .yang.orig. + + + ## 5.8. rpc rebuild-package + --------------------------- + + Rebuild the NED package directly from within NSO. This invokes the gnu make internally. + + Input arguments: + + - verbose + + Print the full build output also for successful builds (otherwise only printed on errors). + + + - profile + + Apply a certain build profile. + + + - filter scope dir + + Directory containing one or many xml file representing the wanted scope. + + + - filter trim-schema method (default patch) + + Select method to be used for trimming. + + deviate - Trim by creating a YANG deviation file containing all selected nodes. + + patch - Trim by patching the YANG models and remove all selected nodes from them before + they are being compiled. + + + Either of: + + - filter trim-schema nodes + + List of nodes to trim. Use one of the pre-defined top node names. Alternatively, specify a + custom xpath to trim (prefix is mandatory on each element in the path). + + OR: + + - filter trim-schema all-unused + + Trim all currently unused nodes in the schema. This means all config nodes that are + currently not populated in CDB. + + OR: + + - filter trim-schema nodes-from-file (default /tmp/nedcom-trim-schema-nodes.txt) + + Specify a path to a custom file to be used for trimming nodes. The file shall contain + schema paths, including relevant prefixes to all nodes to be trimmed. One schema path per + line. + + OR: + + - filter trim-schema custom-deviation-file + + Specify a path to a custom YANG deviation file to be used for trimming the schema. The + file shall comply to the standard for deviation files and contain paths to all nodes to be + trimmed from the schema. + + OR: + + + - filter auto-config dir + + Directory containing the files used for auto-config filtering. The following files must be + present: before.xml and after.xml. + + + - filter auto-config file (default /tmp/nedcom-auto-deviations.yang) + + Name of auto generated deviation file. + + + - ned-id major + + Set a custom major number in the generated ned-id. + + + - ned-id minor + + Set a custom minor number in the generated ned-id. + + + - ned-id suffix + + Set a custom suffix in the generated ned-id. + + + - additional-build-args + + Additional arguments to pass to build(make) commands. + + + ## 5.9. rpc show-default-local-dir + ---------------------------------- + + Show the path to the default directory where the YANG files are to be copied. I.e /src/yang. + + No input arguments + + + ## 5.10. rpc show-loaded-schema + ------------------------------- + + Display the schema currently built into the NED package. Each node will by default be listed with + a schema path. + + Input arguments: + + - scope (default all) + + Select the scope for the nodes that will be listed. + + all - Display all nodes in the schema. This is the default. + + used - Display only the config nodes in use, i.e currently populated in CDB. + + unused - Display only the config nodes that are not in use. + + + - count + + Count the nodes and return the sum instead of the full list of nodes. + + + - root-paths + + Specify root paths for which nodes shall be listed or counted. Only nodes with a schema path + starting any of the specified roots will then be processed. + + + - details + + Display schema details like must/when expression, leafrefs and leafref targets. + + + - config (default true) + + Set to false to display non config nodes in the schema. Note: scope will in this case be + 'all'. + + The NED does not support live-status actions. + + +# 6. Built in live-status show +------------------------------ + + Accedian-skylight_rc NED supports several live-status TTL-based data, mainly denoted in the yang model by the presence of 'config false' annotation in the yang nodes. + + Here are some examples of nodes that supports live-status data (note: since this NED is 3PY yang, the following list is not exhaustive): + ``` + /service-endpoints/service-endpoint/status + /sessions/session/service-endpoints/session-protocol//reports + /sessions/session/status + ``` + + Before using live-status feature of the accedian-skylight_rc NED, several ned-settings have to be configured: + ``` + ned-settings accedian-skylight_rc general live-status restore-namespaces-and-identityrefs true + ned-settings accedian-skylight_rc restconf live-status append-content-nonconfig-query true + ``` + + After commiting the above ned-settings, live-status data can be fetched by executing commands similar as the following: + ``` + admin@ncs# show devices device live-status sessions + live-status sessions session NED_Test_Session_01 + service-endpoints ANT2 + service-endpoints SFP + live-status sessions session SCALE_SOAK_ID-_ip9wucu3pZBIPuPJxo + service-endpoints ei_867a034f-20ea-4b01-b48f-e66193dc69cb_default + service-endpoints ei_ff89a007-a259-44f5-9422-beaef511e6f0_default + live-status sessions session SCALE_SOAK_ID-eCR1JciOGVSJuvIo_F97ZVpFFtEHYCRxsKzV26YW-HNo1JjSeh2xnSBl453mqU1cvIXASa7Y2ODSoFI + service-endpoints ei_867a034f-20ea-4b01-b48f-e66193dc69cb_default + service-endpoints ei_ff89a007-a259-44f5-9422-beaef511e6f0_default + live-status sessions session SCALE_SOAK_ID-eN_xIMVkSbTPPImCj0IKSiZ7XxzbCvr6j + service-endpoints ei_867a034f-20ea-4b01-b48f-e66193dc69cb_default + service-endpoints ei_ff89a007-a259-44f5-9422-beaef511e6f0_default + live-status sessions session SCALE_SOAK_ID-yok3eXFEt_5Km8DceRqY + service-endpoints ei_867a034f-20ea-4b01-b48f-e66193dc69cb_default + service-endpoints ei_ff89a007-a259-44f5-9422-beaef511e6f0_default + ``` + + Above command will fetch live-status data for all sessions. The following example shows how to fetch data for a particular session: + ``` + admin@ncs# show devices device live-status sessions session SESS_ECHO01 + live-status sessions session SESS_ECHO01 + service-endpoints SEP_ECHO1 + status status stopped + admin@ncs# + ``` + + +# 7. Limitations +---------------- + + Accedian Skylight have strong limitations related to service-endpoints creation. This is because the server handles the service-endpoint management. + It is not possible to create a service-endpoint from scratch, however, it is possible to delete an existing service-endpoint and to re-create it with the same id. + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - Access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings accedian-skylight_rc logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. A detailed use case description, with details like: + - Data of interest + - The kind of operations to be used on the data. Like: 'read', 'create', 'update', 'delete' + and the order of the operation + - Device APIs involved in the operations (For example: REST URLs and payloads) + - Device documentation describing the operations involved + + 2. Run sync-from # devices device dev-1 sync-from (if relevant) + + 3. Attach the raw trace to the ticket (if relevant) + + 4. Access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc. + + +# 9. How to rebuild a NED +-------------------------- + + Check the README-rebuild.md file, chapter 1.3, for more information. + + +# 10. NED connection methods +---------------------------- + +The NED has the following authentication options thoward the device: mTLS +Note: it is highly recommeded to use 'ned-settings accedian-skylight_rc connection ssl accept-any false', but for this, the server certificate needs to be valid and trusted by a CA. +If it is not the case, then set accept-any to true: + +``` +ned-settings accedian-skylight_rc connection ssl accept-any true +``` + +For the mTLS mode, use the following ned-settings; + +``` +devices device accsky-1 + address + port + authgroup default + device-type generic ned-id accedian-skylight_rc + state admin-state unlocked + ned-settings accedian-skylight_rc connection ssl accept-any false + ned-settings accedian-skylight_rc connection ssl certificate + ned-settings accedian-skylight_rc connection ssl mtls client certificate + ned-settings accedian-skylight_rc connection ssl mtls client private-key + ned-settings accedian-skylight_rc live-status time-to-live 15 + ned-settings accedian-skylight_rc restconf url-base "/restconf" + ned-settings accedian-skylight_rc restconf model-discovery disabled + ned-settings accedian-skylight_rc restconf profile accedian-skylight_rc + ned-settings accedian-skylight_rc connection authentication method none + ned-settings accedian-skylight_rc restconf notif automatic-stream-discovery enabled + ned-settings accedian-skylight_rc restconf notif preferred-encoding json + ned-settings accedian-skylight_rc restconf capability-discovery disabled + ned-settings accedian-skylight_rc restconf config gather-updates-into-single-patch true + ned-settings accedian-skylight_rc restconf config update-method put + +``` + +Note: the certificates format (eg server/client cert/key) shall be entered in DER Base64 format, which is the same as the PEM format but without the banners \"----- BEGIN CERTIFICATE-----\", \"----- END CERTIFICATE-----\" and newlines (eg '\n') should be escaped (eg replace '\n' with '\\n'). + +Here is an example of how a certificate should look like: + +``` +ned-settings accedian-skylight_rc connection ssl certificate "MIIFrDCCBJSgAwIBAgISA0OTtTWsWgFCAjyUCH4PcMfhMA0GCSqGSIb3DQEBCwUA\nMDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD\nEwJSMzAeFw0yMzAzMDYyMTM3NDJaFw0yMzA2MDQyMTM3NDFaMEIxQDA+BgNVBAMT\nN3BlcmZvcm1hbmNlLnBlcmZvcm1hbmNlLmFnZW50c2xhYi5hbmFseXRpY3MuYWNj\nZWRpYW4uaW8wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDfViyXJXn3\nZWJNciV8qGeiCJhzwOmaUAyhH38UCXpRDQpXhuALuvK9l3BsYImLlEQBxwoAO8ki\nMnRHRZrZk/o28WQ3kHHkQ2A+DRE7AhtJe4d/pe9apqDS2tGRPrZ/2l08v3V4XxyU\n45C7Gkvt3CZ94+meRmfxHQzwqDiFqG8dYOinsxDO815VgjE0FweZhd5sPn4s+JXk\nDMhilDp6VxiBIjanAP5BUldp8TLtItsEz2taIXXwSfSYbmLi2Z5bYAOr9FhtcFOC\nki8lfk9hsDSNsG3tnR+YlzM6ck1Dr10Jam80P2Kf/zzmV92cNeOBn00KCF6i5XjG\nSTq0giI2AAEZAgMBAAGjggKqMIICpjAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYw\nFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFNaK\nacTcDrSbB/lTYKMRGsqNPBt7MB8GA1UdIwQYMBaAFBQusxe3WFbLrlAJQOYfr52L\nFMLGMFUGCCsGAQUFBwEBBEkwRzAhBggrBgEFBQcwAYYVaHR0cDovL3IzLm8ubGVu\nY3Iub3JnMCIGCCsGAQUFBzAChhZodHRwOi8vcjMuaS5sZW5jci5vcmcvMHoGA1Ud\nEQRzMHGCNmFuYWx5dGljczEucGVyZm9ybWFuY2UuYWdlbnRzbGFiLmFuYWx5dGlj\ncy5hY2NlZGlhbi5pb4I3cGVyZm9ybWFuY2UucGVyZm9ybWFuY2UuYWdlbnRzbGFi\nLmFuYWx5dGljcy5hY2NlZGlhbi5pbzBMBgNVHSAERTBDMAgGBmeBDAECATA3Bgsr\nBgEEAYLfEwEBATAoMCYGCCsGAQUFBwIBFhpodHRwOi8vY3BzLmxldHNlbmNyeXB0\nLm9yZzCCAQQGCisGAQQB1nkCBAIEgfUEgfIA8AB2AHoyjFTYty22IOo44FIe6YQW\ncDIThU070ivBOlejUutSAAABhrkSy48AAAQDAEcwRQIhAP1KEfRw2KlUrSkl4s2y\ntD2o8ZNIlcLfyUd3Cgxm2v1CAiBt/lRVvILBJKrLs+fyhfPv48Gwj9p4bOYyym5x\njs+XCwB2AK33vvp8/xDIi509nB4+GGq0Zyldz7EMJMqFhjTr3IKKAAABhrkSy7cA\nAAQDAEcwRQIgd2lLSYzz8Qyy2MsUAH/ug0yFW00/uBsVPHUVMsCw4N0CIQCnnaWR\nohpVUV6wV1yg4wOk2+85Z8kqAKrxs9WOIrOfxjANBgkqhkiG9w0BAQsFAAOCAQEA\nW9fbOIawvoGEes//ddPbO2j0HccFzfljq/mPJG9rC28woCrOu7NA5zKHYZSjjgk6\nHRUtfN51zm3Pww+X7e1zq0LhPQfBXw/XCyFlCTXL3m8pjwJAaoy1NU5qG1ivW4YP\n7zaLq11AjEIkwyBuhz8Zd6lk84eOn4BFxZvSyZmCnbwJ8OTrfyQ2Yy34ameDdcxQ\nPw/ullGjS3u7mLU/DJWLzf0VkAxxSzo4d2CkteEEaaQl5QasjjH4VLA/NsGozjMP\nn0RVXLU/CREWPsRabxh5z3seUwRBzoliAdnCEXLeeDGKxYV27h1jRaAaf3xyioVX\nzBoz0BW3JL5MQF0HHhQYSQ==" +``` + +For more details of other ned-settings, please check README-ned-settings.md + + +# 11. NED Notifications +------------------------ + +The NED supports all notifications streams of Accedian Skylight device. +In order to check the available streams on the device, run the following command: + +``` +admin@ncs# show devices device accsky-1 notifications stream +notifications stream notification-stream/Accedian-service-endpoint:state-change-event + description "This notification is sent when the state of the endpoint changes" + replay-support true + replay-log-creation-time 2024-03-26T16:51:09.89014+00:00 +notifications stream notification-stream/Accedian-service:state-change-event + description "A top level notification providing state change information on the service and it's\nassociated service sessions and service endpoints" + replay-support true + replay-log-creation-time 2024-03-26T13:23:45.883485+00:00 +notifications stream notification-stream/Accedian-session:state-change-event + description "This notification is sent when the state of the session changes" + replay-support true + replay-log-creation-time 2024-03-27T10:35:57.18182+00:00 +notifications stream notification-stream/sal-remote:data-changed-notification + description "Data change notification." + replay-support true +admin@ncs# +``` + +After finding out the usable notifications streams, create new subscriptions in order to subscribe to a certain stream: + +``` +dmin@ncs# config +Entering configuration mode terminal +admin@ncs(config)# devices device accsky-1 +admin@ncs(config-device-accsky-1)# notifications subscription TEST_SUB01 stream notification-stream/Accedian-session:state-change-event local-user admin +admin@ncs(config-subscription-TEST_SUB01)# commit +Commit complete. +admin@ncs(config-subscription-TEST_SUB01)# +``` + +Note: once the subscription configuration is commited, the notification stream monitoring is started automatically and notifications will be received. + +To check for received notifications: + +``` +admin@ncs(config-subscription-TEST_SUB01)# +admin@ncs# show devices device accsky-1 notifications received-notifications +notifications received-notifications notification 2024-03-27T16:25:20.025481+00:00 0 + user admin + subscription TEST_SUB01 + stream notification-stream/Accedian-session:state-change-event + received-time 2024-03-27T16:25:20.15546+00:00 + data acdses:state-change-event session-id bt_demo_Session_ID_2 + data acdses:state-change-event status Stopped +admin@ncs# show devices device accsky-1 notifications received-notifications +notifications received-notifications notification 2024-03-27T16:25:20.025481+00:00 0 + user admin + subscription TEST_SUB01 + stream notification-stream/Accedian-session:state-change-event + received-time 2024-03-27T16:25:20.15546+00:00 + data acdses:state-change-event session-id bt_demo_Session_ID_2 + data acdses:state-change-event status Stopped +admin@ncs# +``` + +Stopping and starting a notification subscription: + +``` +admin@ncs# config +admin@ncs(config)# devices device accsky-1 notifications subscription TEST_SUB01 +admin@ncs(config-subscription-TEST_SUB01)# disconnect +admin@ncs(config-subscription-TEST_SUB01)# reconnect +admin@ncs(config-subscription-TEST_SUB01)# +``` diff --git a/accedian-spp/README-ned-settings.md b/accedian-spp/README-ned-settings.md new file mode 100644 index 00000000..1e5e2742 --- /dev/null +++ b/accedian-spp/README-ned-settings.md @@ -0,0 +1,479 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/accedian-spp/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/accedian-spp/ + device + /ncs:/device/devices/device:/ned-settings/accedian-spp/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings accedian-spp + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings accedian-spp + 2. developer + 3. proxy + 4. connection + 5. console + 5.1. warning + 5.2. command + 5.3. pattern + 5.4. action + 5.4.1. state + 6. logger + 7. live-status + 7.1. auto-prompts + ``` + + +# 1. ned-settings accedian-spp +------------------------------ + + + - extended-parser (default auto) + + Make the accedian-spp NED handle CLI parsing (i.e. transform the running-config from the + device to the model based config tree). + + disabled - Load configuration the standard way. + + turbo-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO using maapi + setvalues call. + + turbo-xml-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO in XML + format. + + robust-mode - Makes the NED filter the configuration so that unmodeled content is removed + before being passed to the NSO CLI-engine. This protects against + configuration ending up at the wrong level when NSO CLI parser fallbacks + (which potentially can cause following config to be skipped). + + auto - Uses turbo-mode when available, will use fastest availablemethod to load + data to NSO. + + + - trans-id-from-conf-changes (default false) + + EXPERIMENTAL. Enables trans-id fetching with the + 'configuration changes' command sent to device. + + +# 2. ned-settings accedian-spp developer +---------------------------------------- + + Contains settings used by the NED developers. + + + - developer load-from-file + + Make the NED load a file containing raw device config when doing sync-from. Does only work on + NETSIM targets. + + + - developer commit-wait-time (default 1000) + + This ned-setting is used to add a sleep timer for the NED + after sending a commit to the device. This is useful to give + the device a small break to parse whatever config it has + been given. Default setting is 1000 ms (1 second). + Netsim device will not take this ned-setting into account. + + + - developer delete-remote-devices-wait-time (default 1000) + + This ned-setting is used to add a sleep timer for the NED + after sending a 'remote-devices delete ' command to the device. + This is useful to avoid false errors like: 'Error: Filter in use', + that occurs when a filter is deleted immediately after its related + remote-devices context was deleted. + Default setting is 1000 ms (1 second). + Netsim device will not take this ned-setting into account. + + + - developer progress-verbosity (default debug) + + Maximum NED verbosity level which will get written in devel.log file. + + disabled - disabled. + + normal - normal. + + verbose - verbose. + + very-verbose - very-verbose. + + debug - debug. + + + - developer platform model + + Override device model name/number. + + + - developer platform name + + Override device name. + + + - developer platform version + + Override device version. + + +# 3. ned-settings accedian-spp proxy +------------------------------------ + + Configure NED to access device via a proxy. + + + - proxy remote-connection + + Connection type between proxy and device. + + ssh - ssh. + + telnet - telnet. + + serial - serial. + + + - proxy remote-address + + Address of host behind the proxy. + + + - proxy remote-port + + Port of host behind the proxy. + + + - proxy remote-name + + User name on the device behind the proxy. + + + - proxy remote-password + + Password on the device behind the proxy. + + + - proxy proxy-prompt + + Prompt pattern on the proxy host before connecting to device. + + + - proxy remote-ssh-args + + Additional arguments used to establish proxy connection. + + +# 4. ned-settings accedian-spp connection +----------------------------------------- + + Configure settings specific to the connection between NED and device. + + + - connection number-of-retries (default 0) + + Configure max number of retries the NED will try to connect to the device before giving up. + Default 0. + + + - connection time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + + - connection ssh client + + Configure the SSH client to use. Relevant only when using the NED with NSO 5.6 or later. + + ganymed - The legacy SSH client. Used on all older versions of NSO. + + sshj - The new SSH client with support for the latest crypto features. This is the default + when using the NED on NSO 5.6 or later. + + + - connection ssh host-key known-hosts-file + + Path to openssh formatted 'known_hosts' file containing valid host keys. + + + - connection ssh host-key public-key-file + + Path to openssh formatted public (.pub) host key file. + + + - connection ssh auth-key private-key-file + + Path to openssh formatted private key file. + + + - connection commands meta-data + + Change the default connector. Default 'ned-connector.json'. + + + - connection commands initial-action + + Interactor action used to initialize a connection. + + + - connection commands awaken-console + + Command sent to awaken console during connection. + + + - connection commands send-delay (default 0) + + Delay in ms before sending a command during connection. + + + - connection commands expect-timeout (default 60000) + + Default limit in ms for waiting for command response. + + +# 5. ned-settings accedian-spp console +-------------------------------------- + + Settings used while interacting with a device. + + + - console ignore-errors (default false) + + Flag indicating if errors should be ignored. + + + - console ignore-warnings (default false) + + Flag indicating if warnings should be ignored. + + + - console ignore-retries (default false) + + Flag indicating if retries should be ignored. + + + - console max-retries (default 100) + + Maximum number of retries of a command. + + + - console retry-delay (default 1000) + + Number of ms before retrying a command. + + + - console send-delay (default 0) + + Enable delay before sending commands. + + + - console expect-timeout (default 60000) + + Set default timeout for sending commands. + + + - console chunk-size (default 1) + + Enable executing commands in chunks. + + + - console line-feed + + Overwrites default line-feed character. + + +## 5.1. ned-settings accedian-spp console extension warning +----------------------------------------------------------- + + Device warning regex entry list. Use to ignore warnings/errors etc. + + - console extension warning + + - warning + + Warning regular expression, e.g. vlan.* does not exist.*. + + +## 5.2. ned-settings accedian-spp console extension command +----------------------------------------------------------- + + Extend available commands to send. + + - console extension command + + - name + + Key id of the command. + + - data + + Command. + + +## 5.3. ned-settings accedian-spp console extension pattern +----------------------------------------------------------- + + Extend available patterns to expect. + + - console extension pattern + + - name + + Key id of the pattern. + + - data + + A regular expression. + + +## 5.4. ned-settings accedian-spp console extension action +---------------------------------------------------------- + + Extend available actions to perform. + + - console extension action + + - name + + A name for the action. + + - init + + Command sent to intialize action. + + - flush + + Flush device buffer once action is completed. + + +### 5.4.1. ned-settings accedian-spp console extension action state +------------------------------------------------------------------- + + Extend state machine with answers/questions to handle. + + - state + + - pattern + + Regular expression indicating action required. + + - method + + Method used to take action. + + reportInfo - reportInfo. + + reportError - reportError. + + reportWarning - reportWarning. + + sendCommand - sendCommand. + + sendSecret - sendSecret. + + sendRetry - sendRetry. + + recoverError - recoverError. + + - argument + + Additional info to method. + + - next (default DONE) + + State once action is taken. + + +# 6. ned-settings accedian-spp logger +------------------------------------- + + Settings for controlling logs generated. + + + - logger level (default debug) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - logger java (default true) + + Toggle logs to be added to ncs-java-vm.log. + + +# 7. ned-settings accedian-spp live-status +------------------------------------------ + + Configure NED settings related to live-status. + + + - live-status time-to-live (default 50) + + Define time-to-live for data fetched from the device via live-status.(default 50). + + +## 7.1. ned-settings accedian-spp live-status auto-prompts +---------------------------------------------------------- + + Generally the command output parsing halts when the NED detects + an operational or config prompt, however sometimes the command + requests additional input, "answer(s)" to questions. + + Use "EXIT" in answer to Halt parsing and return output + + Use this ned-setting to create auto-prompt question and answer, + example with "remote-devices factory-reset " command below: + + # devices device sppdevicename ned-settings accedian-spp live-status auto-prompts test question ".*Are you sure you want to factory-reset this device \? \(yes/no\):" answer yes + + - live-status auto-prompts + + - id + + List id, any string. + + - question + + Device question, regular expression. + + - answer + + Answer to device question. + + diff --git a/accedian-spp/README.md b/accedian-spp/README.md new file mode 100644 index 00000000..8a41c24d --- /dev/null +++ b/accedian-spp/README.md @@ -0,0 +1,1063 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + ``` + + +# 1. General +------------ + + This document describes the accedian-spp NED. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | - | + | | | | + | check-sync | yes | - | + | | | | + | partial-sync-from | yes | - | + | | | | + | live-status actions | yes | - | + | | | | + | live-status show | yes | - | + | | | | + | load-native-config | no | - | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Skylight sensor: control | VCX_19.12.0_219 | Linux- | - | + | | 89 | based | | + | | | | | + | Skylight sensor: control | VCX_22.12.2_254 | Linux- | - | + | | 56 | based | | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--accedian-spp-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-accedian-spp-1.0.1.signed.bin + > ./ncs-6.0-accedian-spp-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-accedian-spp-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-accedian-spp-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `accedian-spp-.`: + + ``` + > tar xfz ncs-6.0-accedian-spp-1.0.1.tar.gz + > ls -d */ + accedian-spp-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package accedian-spp-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package accedian-spp-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-accedian-spp-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package accedian-spp-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/accedian-spp-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-accedian-spp-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-accedian-spp-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install accedian-spp-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-accedian-spp-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-accedian-spp-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id accedian-spp-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-accedian-spp-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings accedian-spp logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings accedian-spp logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.accedianspp \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings accedian-spp logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings accedian-spp logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + **SSHJ DEBUG LOGGING** + For issues related to the ssh connection it is often useful to enable full logging in the SSHJ ssh client. + This will make SSHJ print additional log entries in `$NSO_RUNDIR/logs/ncs-java-vm.log`: + +``` +admin@ncs(config)# java-vm java-logging logger net.schmizz.sshj level level-all +admin@ncs(config)# commit +``` + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + For instance, create a new user: + + ``` + admin@ncs(config)# devices device dev-1 config + admin@ncs(config-config)# user drneduser first Ned last Doctor phone 123456978 email doctor.ned@email.com + ``` + + See what you are about to commit: + + ``` + admin@ncs(config-if)# commit dry-run outformat native + device + name dev-1 + data user add drneduser first Ned last Doctor phone 123456978 email doctor.ned@email.com + ``` + + Commit new configuration in a transaction: + + ``` + admin@ncs(config-if)# commit + Commit complete. + ``` + + Verify that NCS is in-sync with the device: + + ``` + admin@ncs(config-if)# devices device dev-1 check-sync + result in-sync + ``` + + Compare configuration between device and NCS: + + ``` + admin@ncs(config-if)# devices device dev-1 compare-config + admin@ncs(config-if)# + ``` + + Note: if no diff is shown, supported config is the same in + NCS as on the device. + + +# 5. Built in live-status actions +--------------------------------- + + The NED has support for subset of native accedian spp exec commands residing + under device live-status. Presently, the following commands are supported: + ``` + admin@ncs# devices device live-status exec ? + Possible completions: + any Run any command on the device + cfm-show-mep-status get cfm show mep status + get-policies-vcx-and-device get count/number of policies existing on the VCX Controller or/and on specific ANT/NANO device + set-password set password for user + show Execute show commands + ``` + To execute a command, run it in NCS exec mode like this: + Example: + ``` + admin@ncs# devices device live-status exec any "remote-devices show" + + ``` + If the command triggers a prompt like the example below: + ``` + admin@ncs# devices device live-status exec any "remote-devices factory-reset " + ``` + This will only work properly if the ned-setting live-status auto-prompt + is set correctly, see README-ned-settings.md chapter 7.1. + + NOTE: when using multiple commands user need to separate commands with " ; " + + 5.1 Execute set user password + ------------------------------ + As user password is not config that is shown on device, + there is an action implemented that can simplify setting + a password for specific user. + + Example: + ``` + admin@ncs# devices device live-status exec set-password user password + ``` + + 5.2 Get the cfm show mep status of a remote device + -------------------------------------------------- + Example: + ``` + admin@ncs# devices device live-status exec cfm-show-mep-status context status + ``` + + 5.3 Get the number of policies existing on the VCX Controller or on a specific ANT/NANO device + ---------------------------------------------------------------------------------------------- + Examples: + + To fetch policies on a VCX Controller: + + ``` + admin@ncs(config)# devices device live-status exec get-policies-vcx-and-device context + result 4 + ``` + + To fetch policies on a specific ANT/NANO device: + + ``` + admin@ncs(config)# devices device live-status exec get-policies-vcx-and-device context device + result 2 + ``` + + 5.4 The 'inventory add' and 'inventory clear' commands + ------------------------------------------------------ + + These commands affect the action that the user wants to do on the ANT modules. This is the action that takes place in the VCX inventory. + + Examples: + Checking the inventory status: + + ``` + admin@ncs(config)# devices device live-status exec any inventory show remote-devices + result + inventory show remote-devices -> + Serial System description IP address FW version Name + ----------------- -------------------------- --------------- ---------------------- ------------ + C404-9787 ANT-1000-AX-R-AC 0.0.0.0 rAFF_PMON_20.11_8134 C404-9787 + + AFB5CAAD-C1D9-451E-8376-8D80A59B5D96: + ``` + + Checking the remote-devices status: + + ``` + admin@ncs(config)# devices device live-status exec any remote-devices show + result + remote-devices show -> + Instance Remote Device Name MAC address Linked Auth Admin State State + -------- ------------------------------ ----------------- ------------ ---- ----------- ------------ + + AFB5CAAD-C1D9-451E-8376-8D80A59B5D96: + ``` + + Once a device is discovered and available under the inventory section, then it can be transferred to the remote-devices list as below: + + ``` + admin@ncs(config)# devices device live-status exec any inventory add remote-device serial-number C404-9787 override-config yes + result + inventory add remote-device serial-number C404-9787 override-config yes -> + AFB5CAAD-C1D9-451E-8376-8D80A59B5D96: + ``` + + Checking if it was added under the remote-devices list: + + ``` + admin@ncs(config)# devices device live-status exec any remote-devices show + result + remote-devices show -> + Instance Remote Device Name MAC address Linked Auth Admin State State + -------- ------------------------------ ----------------- ------------ ---- ----------- ------------ + 1 C404-9787 00:15:AD:3C:AF:0C Initializing Yes OOS Connected + + AFB5CAAD-C1D9-451E-8376-8D80A59B5D96: + ``` + + The “inventory clear” command will only clear the inventory for 2 seconds and then the ANT will reappear. + If the ANT is on the network, the "clear" command will make the ANT disappear, but then it will reappear, which is normal. + + ``` + admin@ncs(config)# devices device live-status exec any inventory clear remote-devices + result + inventory clear remote-devices -> + AFB5CAAD-C1D9-451E-8376-8D80A59B5D96: + ``` + + +# 6. Built in live-status show +------------------------------ + + NONE + + +# 7. Limitations +---------------- + + 7.1 remote-devices override-config not configurable + --------------------------------------------------- + UPDATE: "override-config no" is for now not supported by the NED. + + The leaf "override-config" is only configurable to the value "yes". + + 7.2 remote-devices context mode + ------------------------------- + Updated context mode. The remote-devices list is now modeled + and used as a specific mode by the NED. This will significantly + improve the speed when doing larger commits, since it won't + enter the specific context mode multiple times. + + Note: this is an backward incompatible change. Some scripts + or user config files could have to be changed. + + Example of how to config a remote-device and a virtual-connection: + + ``` + remote-devices REMOTE-DEVICE-NAME + layer2-interface None override-config yes type Ant2Combo flex-monitor disable device-mac-addr 00:15:AD:21:FB:88 default-traffic-fwd permit + virtual-connection vca-vlan VC-NAME tp-a-vlan-stack-size all-to-one tp-a-port UNI tp-z-vlan-stack-size 1 vlan-tp-z-1-tpid 0x8100 vlan-tp-z-1-id 100 tp-z-port NNI tp-a1z1-pcp-mapping drnedcosprof + exit + ``` + + 7.3 inventory remote-devices + ---------------------------- + Since the device module "inventory" is not controlled by the device itself, + the config it is holding is dependent on what is discovered in the + device network, it is not valid to model for the NED at this stage. + + Beware that if the user adds a remote-device by issuing a + inventory add remote-devices + with a live-status exec, the NED will need a sync-from + to fetch the newly created remote-devices instance along with its + sub-config. + + 7.4 port auto-config + -------------------- + When a remote-devices instance is configured on the device, + four corresponding port instances will be generated/auto-configured. + + Hence, when this config has to be similarly mirrored on the NED side + to stay in sync. + + 7.5 port/state availability on device + -------------------------------------- + As the "state" of a port instance can only be changed for some + cases, the NED will now only read the state value from device + if the port instance is connected through "SFP-2", example: + + port show configuration C404-9787-UNI + " + Port name: C404-9787-UNI + Connector : SFP-2 + ... + " + + There are currently no restrictions from NED side when sending + config towards the device regarding this "state" leaf. + + 7.6 remote-devices/virtual-connection/vca-vlan auto-config + ---------------------------------------------------------- + When creating a instance of this node with config that + has values for all three TP Z's, two TP A's, along with + "contain-tunnel yes", the device will automatically create + a new instance of vca-vlan. This instance will be named + with the same name as the original one, only with a "VCA-" + as prefix and "-Z" as suffix. This has to be mirrored from + the NED side. But the NED will filter out these lines when + committing to device, since an error could be thrown if, for + example, try to add an instance which is already created. + + Example: + + When creating the following: + virtual-connection vca-vlan TESTNAME tp-a-vlan-stack-size 2 tp-a-port UNI tp-z-vlan-stack-size 3 vlan-tp-z-1-tpid 0x8100 vlan-tp-z-1-id 100 tp-z-port NNI tp-a1z1-pcp-mapping 8P0D-8P0D vlan-tp-a-1-tpid 0x8100 vlan-tp-a-1-id 100 vlan-tp-z-2-id 200 vlan-tp-z-2-tpid 0x88a8 vlan-tp-a-2-tpid 0x88a8 vlan-tp-a-2-id 200 vlan-tp-z-3-id 400 vlan-tp-z-3-tpid 0x9100 contain-tunnel yes tp-z1a1-pcp-mapping 8P0D-8P0D + + Will have to be mirrored by: + virtual-connection vca-vlan VCA-TESTNAME-Z tp-a-vlan-stack-size 2 tp-a-port UNI tp-z-vlan-stack-size 3 tp-z-port NNI tp-a1z1-pcp-mapping none vlan-tp-z-3-id 400 vlan-tp-z-3-tpid 0x9100 contain-tunnel yes tp-z1a1-pcp-mapping none + + What NED will send to device: + virtual-connection add vca-vlan TESTNAME tp-a-vlan-stack-size 2 tp-a-port UNI tp-z-vlan-stack-size 3 vlan-tp-z-1-tpid 0x8100 vlan-tp-z-1-id 100 tp-z-port NNI tp-a1z1-pcp-mapping 8P0D-8P0D vlan-tp-a-1-tpid 0x8100 vlan-tp-a-1-id 100 vlan-tp-z-2-id 200 vlan-tp-z-2-tpid 0x88a8 vlan-tp-a-2-tpid 0x88a8 vlan-tp-a-2-id 200 vlan-tp-z-3-id 400 vlan-tp-z-3-tpid 0x9100 contain-tunnel yes tp-z1a1-pcp-mapping 8P0D-8P0D + + Now the NED and device will still be in sync. + + NOTE: Short example that does not include any extra lines + that would be needed here like remote-devices context mode, + or bandwidth-regulator-set auto-configuration. + + 7.7 virtual-connection/vca-vlan/bandwidth-regulator-envelope/rank + ----------------------------------------------------------------- + This is currently modeled as a leaf, this requires + that when configuring it for multiple + bandwidth-regulator(s) the user has to quote the + values. Do not use commas, as the NED will not have + those when reading from device, and the NED will + automatically append those when sending config to + device. + + Example: + remote-devices REMOTE-D-NAME + virtual-connection vca-vlan VCA-VLAN-NAME bandwidth-regulator-envelope direction AtoZ rank "bwregulator1 bwregulator2 bwregulator4 bwregulator6 bwregulator3 bwregulator5" + ! + + 7.8 device prompt + ---------------------------------------------------------- + Currently the NED has a requirement that the device + prompt has a "-" (hyphen-sign) in it. If users encounter + a scenario where this is not the case, submit a bug + and the NED team will look in to it. + + Examples of current acceptable prompts: + 1. A631E32D-98C3-45D0-8D9C-42F6B78F885F: + 2. VCX-DUMMYPROMPT: + + 7.9 loopback name and device-name + ---------------------------------------------------------- + The VCX device requires a remote-device or port to be entered + before the loopback name (list key) when creating an instance. + Example: + loopback add + + To "mirror" this behavior, the NED requires remote-device + to be entered before specifiying loopback name, both when creating + and modifying. When deleting an instance, only loopback name should + be used. + Example NCS_CLI: + create/modify: + loopback + delete: + no loopback + + The above may not be easily understood with only tab completion + in NCS_CLI. + + 7.10 "edit"-only nodes and undeletable configs + ---------------------------------------------------------- + Some config may only be modified in device, no other operations + allowed. At least as of now it's not known how to delete them + without doing a complete device reset. + + This may make the NED unable to rollback certain scenarios, + where the config is set for the first time on device. + + 7.11 modify port "auto-nego" and "advertisement" parameters + ----------------------------------------------------------- + The "advertisement" and "auto-nego" parameters must be always provided togheter(even when the new values are similar to the old ones). + It is necessary to keep the sync between device's CDB and the accedian device. + + To be able to change the "advertisement", the "auto-nego" must be enabled first. + To set "auto-nego" back to disabled, the "advertisement" must be provided before "auto-nego disabled". + + Example: given the initial state below: + + ``` + port rem-device1-NNI lldp-state enable advertisement 1G-FD auto-nego disable speed 1G + ``` + + 1. enable "auto-nego" + + ``` + port rem-device1-NNI auto-nego enable advertisement 100M-FD,1G-FD speed 100M + + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name dev-1 + data port edit rem-device1-NNI lldp-state enable auto-nego enable advertisement 100M-FD,1G-FD speed 100M + } + } + admin@ncs(config-config)# commit + Commit complete. + admin@ncs(config-config)# compare-config + admin@ncs(config-config)# + ``` + + 2. disable "auto-nego" + + ``` + port C404-9787-NNI lldp-state enable advertisement 1G-FD auto-nego disable speed 1G + + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name dev-1 + data port edit C404-9787-NNI lldp-state enable advertisement 1G-FD auto-nego disable speed 1G + } + } + admin@ncs(config-config)# commit + Commit complete. + admin@ncs(config-config)# compare-config + admin@ncs(config-config)# + ``` + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings accedian-spp logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings accedian-spp logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` diff --git a/actelis-ead/README-ned-settings.md b/actelis-ead/README-ned-settings.md new file mode 100644 index 00000000..fee0db53 --- /dev/null +++ b/actelis-ead/README-ned-settings.md @@ -0,0 +1,197 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/actelis-ead/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/actelis-ead/ + device + /ncs:/device/devices/device:/ned-settings/actelis-ead/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings actelis-ead + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings actelis-ead + 2. proxy + 3. connection + 4. live-status + 5. logger + ``` + + +# 1. ned-settings actelis-ead +----------------------------- + + + - extended-parser (default auto) + + Make the NED handle CLI parsing (i.e. transform the running-config from the device to the + model based config tree). + + disabled - Load configuration the standard way. + + turbo-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO using maapi + setvalues call. + + turbo-xml-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO in XML + format. + + robust-mode - Makes the NED filter the configuration so that unmodeled content is removed + before being passed to the NSO CLI-engine. This protects against + configuration ending up at the wrong level when NSO CLI parser fallbacks + (which potentially can cause following config to be skipped). + + auto - Uses turbo-mode when available, will use fastest availablemethod to load + data to NSO. If NSO doesn't support data-loading from CLI NED, robust-mode + is used. + + +# 2. ned-settings actelis-ead proxy +----------------------------------- + + Configure NED to access device via a proxy. + + + - proxy remote-connection + + Connection type between proxy and device. + + ssh - ssh. + + telnet - telnet. + + serial - serial. + + + - proxy remote-address + + Address of host behind the proxy. + + + - proxy remote-port + + Port of host behind the proxy. + + + - proxy remote-name + + User name on the device behind the proxy. + + + - proxy remote-password + + Password on the device behind the proxy. + + + - proxy proxy-prompt + + Prompt pattern on the proxy host before connecting to device. + + + - proxy remote-ssh-args + + Additional arguments used to establish proxy connection. + + +# 3. ned-settings actelis-ead connection +---------------------------------------- + + Configure settings specific to the connection between NED and device. + + + - connection number-of-retries (default 0) + + Configure max number of retries the NED will try to connect to the device before giving up. + Default 0. + + + - connection time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + + - connection commands meta-data + + Change the default connector. Default 'ned-connector.json'. + + + - connection commands initial-action + + Interactor action used to initialize a connection. + + + - connection commands awaken-console + + Command sent to awaken console during connection. + + + - connection commands send-delay (default 0) + + Delay in ms before sending a command during connection. + + + - connection commands expect-timeout (default 60000) + + Default limit in ms for waiting for command response. + + +# 4. ned-settings actelis-ead live-status +----------------------------------------- + + Configure NED settings related to live-status. + + + - live-status time-to-live (default 50) + + Define time-to-live for data fetched from the device via live-status.(default 50). + + +# 5. ned-settings actelis-ead logger +------------------------------------ + + Settings for controlling logs generated. + + + - logger level (default info) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - logger java (default false) + + Toggle logs to be added to ncs-java-vm.log. + + diff --git a/actelis-ead/README.md b/actelis-ead/README.md new file mode 100644 index 00000000..be45ca7c --- /dev/null +++ b/actelis-ead/README.md @@ -0,0 +1,792 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + ``` + + +# 1. General +------------ + + This document describes the actelis-ead NED. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | Simulate device from NED yang model | + | | | | + | check-sync | yes | | + | | | | + | partial-sync-from | yes | | + | | | | + | live-status actions | yes | Check README.md section 'Built in live-status actions' | + | | | | + | live-status show | yes | Check README.md section 'Built in live-status show' | + | | | | + | load-native-config | no | | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + Custom NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | proxy-connection | yes | Supports up to 1 proxy jumps | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | ML638 | 7.45-522R66118E | | | + | | | | | + | ML600 | 8.20/57 | | | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--actelis-ead-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-actelis-ead-1.0.1.signed.bin + > ./ncs-6.0-actelis-ead-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-actelis-ead-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-actelis-ead-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `actelis-ead-.`: + + ``` + > tar xfz ncs-6.0-actelis-ead-1.0.1.tar.gz + > ls -d */ + actelis-ead-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package actelis-ead-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package actelis-ead-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-actelis-ead-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package actelis-ead-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/actelis-ead-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-actelis-ead-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-actelis-ead-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install actelis-ead-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-actelis-ead-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-actelis-ead-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id actelis-ead-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-actelis-ead-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings actelis-ead logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings actelis-ead logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.ead \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings actelis-ead logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings actelis-ead logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + **SSHJ DEBUG LOGGING** + For issues related to the ssh connection it is often useful to enable full logging in the SSHJ ssh client. + This will make SSHJ print additional log entries in `$NSO_RUNDIR/logs/ncs-java-vm.log`: + +``` +admin@ncs(config)# java-vm java-logging logger net.schmizz.sshj level level-all +admin@ncs(config)# commit +``` + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + ``` + admin@ncs(config)# devices device dev-1 config + admin@ncs(config-config)# vlan 3999 name TEST ports ETH-1&HSL-1 sports ETH-1 + admin@ncs(config-config)# top + ``` + + See what you are about to commit: + + ``` + admin@ncs(config)# commit dry-run outformat native + native { + device { + name dev-1 + data create vlan 3999 name TEST ports ETH-1&HSL-1 sports ETH-1 + } + } + admin@ncs(config)# + ``` + + Commit new configuration in a transaction: + + ``` + # commit + Commit complete. + ``` + + Verify that NCS is in-sync with the device: + + ``` + # devices device dev-1 check-sync + result in-sync + ``` + + Compare configuration between device and NCS: + + ``` + # devices device dev-1 compare-config + ``` + + Note: if no diff is shown, supported config is the same in NSO as on the device. + + +# 5. Built in live-status actions +--------------------------------- + + NED supports following live-status exec actions commands: + + ``` + admin@ncs# devices device dev-1 live-status exec ? + Possible completions: + any Execute any (multiple) commands on device + ``` + + Example: + + ``` + admin@ncs# devices device dev-1 live-status exec any "show system version" + result + show system version -> + Keyword Description Value + ------------------ ------------------------------------------------------------ ----------- + swversion Current SW major.minor/build version 8.20/57 + ``` + + +# 6. Built in live-status show +------------------------------ + + NED supports following live-status show commands: + + ``` + admin@ncs# show devices device dev-1 live-status? + Possible completions: + card show card info all + port Port info + system System info + + ``` + + Example: + + ``` + admin@ncs# show devices device dev-1 live-status card + live-status card ML600 + admin Enabled + operational "In Service" + actype ML624i + sn ABCDXYZ + swver 8.20/57 + ``` + + +# 7. Limitations +---------------- + + NONE + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings actelis-ead logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings actelis-ead logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` diff --git a/administration/advanced-topics/README.md b/administration/advanced-topics/README.md deleted file mode 100644 index 85db95fe..00000000 --- a/administration/advanced-topics/README.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -description: Deep-dive into advanced NSO concepts. -icon: layer-plus ---- - -# Advanced Topics - diff --git a/administration/advanced-topics/cdb-persistence.md b/administration/advanced-topics/cdb-persistence.md deleted file mode 100644 index 52e7a906..00000000 --- a/administration/advanced-topics/cdb-persistence.md +++ /dev/null @@ -1,69 +0,0 @@ ---- -description: Select the optimal CDB persistence mode for your use case. ---- - -# CDB Persistence - -The Configuration Database (CDB) is a built-in datastore for NSO, specifically designed for network automation use cases and backed by the YANG schema. Since NSO 6.4, the CDB can be configured to operate in one of the two distinct modes: `in-memory-v1` and `on-demand-v1`. - -The `in-memory-v1` mode keeps all the configuration data in RAM for the fastest access time. New data is persisted to disk in the form of journal (WAL) files, which the system uses on every restart to reconstruct the RAM database. But the amount of RAM needed is proportional to the number of managed devices and services. When NSO is used to manage a large network, the amount of needed RAM can be quite large. This is the only CDB persistence mode available before NSO 6.4. - -The `on-demand-v1` mode loads data on demand from the disk into the RAM and supports offloading the least-used data to free up memory. Loading only the compiled YANG schema initially (in the form of .fxs files) results in faster system startup times. This mode was first introduced in NSO 6.4. - -{% hint style="warning" %} -For reliable storage of the configuration on disk, regardless of the persistence mode, the CDB requires that the file system correctly implements the standard primitives for file synchronization and truncation. For this reason (as well as for performance), NFS or other network file systems are unsuitable for use with the CDB - they may be acceptable for development, but using them in production is unsupported and strongly discouraged. -{% endhint %} - -Compared to `in-memory-v1`, `on-demand-v1` mode has a number of benefits: - -* **Faster startup time**: Data is not loaded into memory at startup; only the schema is. -* **Lower memory requirements**: Data is loaded into memory only when needed and offloaded when not. -* **Faster sync of high-availability nodes**: Only subscribed data on the followers is loaded at once. -* **Background compaction**: The compaction process no longer locks the CDB, allowing writes to proceed uninterrupted. - -While the `on-demand-v1` mode is as fast for reads of "hot" data (already in memory) as the `in-memory-v1` mode, reads are slower for "cold" data (not loaded in memory), since the data first has to be read from disk. In turn, this results in a bigger variance in the time that a read takes in the `on-demand-v1` mode, based on whether the data is already available in RAM or not. The variance could express in different ways, for example, by taking a longer time to produce the service mapping or creating a rollback for the first request. To lessen the effect, we highly recommend fast storage, such as NVMe flash drives. - -Furthermore, the two modes differ in the way they internally organize and store data, resulting in different performance characteristics. If sufficient RAM is available, in some cases, `in-memory-v1` performs better, while in others, `on-demand-v1` performs better. One known case where the performance of `on-demand-v1` does not reach that of `in-memory-v1` is deleting large trees of data. But in general, only extensive testing of the specific use case can tell which mode performs better. - -As a rule of thumb, we recommend the `on-demand-v1` mode, as it has typical performance comparable to `in-memory-v1` but has better maintainability properties. However, if performance requirements and testing favor the `in-memory-v1` mode, that may be a viable choice. Discounting the migration time, you can easily switch between the two modes with automatic migration at system startup. - -## Configuring Persistence Mode - -The CDB persistence is configured under `/ncs-config/cdb/persistence` in the `ncs.conf` file. The `format` leaf selects the desired persistence mode, either `on-demand-v1` or `in-memory-v1` (default `in-memory-v1`), and the system automatically migrates the data on the next start if needed. Note that the system will not be available for the migration duration. - -With the `on-demand-v1` mode, additional offloading configuration under `offload` container becomes relevant (`in-memory-v1` keeps all data in RAM and does not perform any offloading). The `offload/interval` specifies how often the system checks its memory consumption and starts the offload process if required. - -During the offloading process, data is evicted from memory: - -1. If the piece of data was last accessed more than `offload/threshold/max-age` ago (the default value of infinity disables this check). -2. The least-recently-used items are evicted until their usage drops below the allowed amount. - -The allowed amount is defined either by the absolute value `offload/threshold/megabytes` or by `offload/threshold/system-memory-percentage`, where the value is calculated dynamically based on the available system RAM. We recommend using the latter unless testing has shown specific requirements. - -The actual value should be adjusted according to the use case and system requirements; there is no single optimal setting for all cases. We recommend you start with defaults and then adjust according to observations. You can enable the new `/ncs-config/cdb/persistence/db-statistics` property to aid you in this task (producing `LOG` files inside the CDB directory), as well as the counters and gauges that are available under `/ncs:metric/sysadmin/*/cdb`. - -## Compaction - -For durability, improved performance, and snapshot isolation, CDB writes in NSO use data structures, such as a write-ahead log (WAL), that require periodic compaction. - -For example, the `in-memory-v1` persistence mode appends a new log entry for each CDB transaction to the target datastore WAL file (`A.cdb` for configuration, `O.cdb` for operational, and `S.cdb` for snapshot datastore). Depending on the size and number of transactions towards the system, these files will grow in size leading to increased disk utilization, longer boot times, and longer initial data synchronization time when setting up a high-availability cluster using this persistence mode. - -Compaction is a mechanism used to reduce the size of the write-ahead logs to a minimum. In `on-demand-v1` mode, it is automatic, non-configurable, and runs in the background without affecting the ongoing transactions. - -But in `in-memory-v1` mode, it works by replacing an existing write-ahead log, which is composed of a number of consecutive transaction logs created in run-time, with a single transaction log representing the full current state of the datastore. From this perspective, a compaction acts similarly to a write transaction towards a datastore. To ensure data integrity, 'write' transactions towards the datastore are not permitted during the time compaction takes place. For this reason, NSO exposes a number of settings to control the compaction process in `in-memory-v1` mode (these have no effect for `on-demand-v1`). - -### Compacting In-Memory CDB - -By default, compaction is handled automatically by the CDB. After each transaction, CDB evaluates whether compaction is required for the affected datastore. - -This is done by examining the number of added nodes as well as the file size changes since the last performed compaction. The thresholds used can be modified in the `ncs.conf` file by configuring the `/ncs-config/compaction/file-size-relative`, `/ncs-config/compaction/file-size-absolute`, and `/ncs-config/compaction/num-node-relative` settings. - -It is also possible to automatically trigger compaction after a set number of transactions by setting the `/ncs-config/compaction/num-transaction` property. - -In the configuration datastore, compaction is by default delayed by 5 seconds when the threshold is reached to prevent any upcoming write transaction from being blocked. If the system is idle during these 5 seconds, meaning that there is no new transaction, the compaction will initiate. Otherwise, compaction is delayed by another 5 seconds. The delay time can be configured in `ncs.conf` by setting the `/ncs-config/compaction/delayed-compaction-timeout` property. - -As compaction may require a significant amount of time, it may be preferable to disable automatic compaction by CDB and instead trigger compaction manually according to specific needs. If doing so, it is highly recommended to have another automated system in place. Automation of compaction can be done by using a scheduling mechanism such as CRON or by using the NCS scheduler. See [Scheduler](../../development/connected-topics/scheduler.md) for more information. - -By default, CDB may perform compaction during its boot process. This may be disabled, if required, by starting NSO with the flag `--disable-compaction-on-start`. - -Additionally, CDB CAPI provides a set of functions that may be used to create an external mechanism for compaction. See `cdb_initiate_journal_compaction()`, `cdb_initiate_journal_dbfile_compaction()`, and `cdb_get_compaction_info()` in [confd\_lib\_cdb(3)](../../resources/man/confd_lib_cdb.3.md) in Manual Pages. diff --git a/administration/advanced-topics/cryptographic-keys.md b/administration/advanced-topics/cryptographic-keys.md deleted file mode 100644 index 09f3211a..00000000 --- a/administration/advanced-topics/cryptographic-keys.md +++ /dev/null @@ -1,183 +0,0 @@ ---- -description: >- - Store strings in NSO that are encrypted and decrypted using cryptographic - keys. ---- - -# Cryptographic Keys - -By using the NSO built-in encrypted YANG extension types `tailf:aes-cfb-128-encrypted-string` or `tailf:aes-256-cfb-128-encrypted-string`, it is possible to store encrypted string values in NSO. See the [tailf\_yang\_extensions(5)](../../resources/man/tailf_yang_extensions.5.md#yang-types-2) man page for more details on the encrypted string YANG extension types. - -## Providing Keys - -NSO supports defining one or more sets of cryptographic keys directly in `ncs.conf` or using an external command. Three methods can be used to configure the keys in `ncs.conf`: - -* External command providing keys under `/ncs-config/encrypted-strings/external-keys`. -* Key rotation under `/ncs-config/encrypted-strings/key-rotation`. -* Legacy (single generation) format: `/ncs-config/encrypted-strings/AESCFB128` and `/ncs-config/encrypted-strings/AES256CFB128` . - -### NSO Installer-Provided Cryptography Keys - -* **Local installation**: Dummy keys are provided in legacy format in `ncs.conf` for development purposes. For deployment, the keys must be changed to random values. Example local installation `ncs.conf` (do not reuse): - - ```xml - - - - 0123456789abcdef0123456789abcdeg - - - 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdeg - - - - ``` -* **System installation**: Random keys are generated in the legacy format stored in `${NCS_CONFIG_DIR}/ncs.crypto_keys`, and read using the `${NCS_DIR}/bin/ncs_crypto_keys` external command as configured in `${NCS_CONFIG_DIR}/ncs.conf`. Example system installation `ncs.conf:` - - ```xml - - - - ${NCS_DIR}/bin/ncs_crypto_keys - ${NCS_CONFIG_DIR}/ncs.crypto_keys - - - - ``` - - Example system installation`ncs.crypto_keys` file (do not reuse): - - ``` - AESCFB128_KEY=40f7c3b5222c1458be3411cdc0899fg - AES256CFB128_KEY=5a08b6d78b1ce768c67e13e76f88d8af7f3d925ce5bfedf7e3169de6270bb6eg - ``` - - For details on using a custom external command to read the encryption keys, see [Encrypted Strings](../../development/connected-topics/encryption-keys.md). - -You can generate a new set of keys, e.g. for use within the `ncs.crypto_keys` file, with the following command (requires `openssl` to be present): - -```sh -#!/bin/sh -cat < - - - 0 - - 0123456789abcdef0123456789abcdeg - - - 3c687d564e250ad987198d179537af563341357493ed2242ef3b16a881dd608g - - - - 1 - - 0123456789abcdef0123456789abcdeh - - - 3c687d564e250ad987198d179537af563341357493ed2242ef3b16a881dd608h - - - - -``` - -External keys that can be rotated must be provided with the initial line `EXTERNAL_KEY_FORMAT=2` and the `generation` within square brackets. Example (do not reuse): - -``` -EXTERNAL_KEY_FORMAT=2 -AESCFB128_KEY[0]=0123456789abcdef0123456789abcdeg -AES256CFB128_KEY[0]=3c687d564e250ad987198d179537af563341357493ed2242ef3b16a881dd608g -AESCFB128_KEY[1]=0123456789abcdef0123456789abcdeh -AES256CFB128_KEY[1]=3c687d564e250ad987198d179537af563341357493ed2242ef3b16a881dd608h -``` - -There is always an active generation: - -* Active generation is the generation in the set of keys currently used to encrypt and decrypt all leafs with an encrypted string type. -* The active generation is persisted. -* If using the legacy method of providing keys in `ncs.conf` or when providing keys using the `/ncs-config/encrypted-strings/key-rotation` method without providing the initial line `EXTERNAL_KEY_FORMAT=2` in the application, the active generation will be `-1`. -* If starting NSO without any previous keys using the `/ncs-config/encrypted-strings/key-rotation` method or the `external-keys` method with the initial line `EXTERNAL_KEY_FORMAT=2`, the highest provided generation will be selected as the active generation. - -For `ncs.conf` details, see the [ncs.conf(5) man page](../../resources/man/ncs.conf.5.md) under `/ncs-config/encrypted-strings`. - -## Key Rotation - -Rotating cryptographic keys means replacing an old cryptographic key with a new one while maintaining the functionality of the encryption and decryption of encrypted string values in NSO. It is a standard practice in cryptography and key management to enhance security and mitigate risks associated with key exposure or compromise.\ -Key rotation helps ensure that sensitive data remains secure over time. It reduces the impact of potential key compromise and adheres to best practices for cryptographic hygiene. Key benefits: - -* If a cryptographic key is compromised, rotating it reduces the amount of data exposed to the attacker since previously encrypted values can be re-encrypted with a new key. -* Regular rotation minimizes the time a single key is in use, thereby reducing the potential damage an attacker could do if they gain access to it. -* Reusing the same key for a prolonged period increases the risk of data correlation attacks (e.g., frequency analysis). Rotation ensures unique keys are used for encrypting strings, reducing this risk. -* Regularly rotating keys helps organizations maintain and test their key management processes. This ensures the system is prepared to handle key management tasks effectively in an emergency. - -To rotate to a new generation of keys and re-encrypt the data: - -1. Always [take a backup](../management/system-management/#backup-and-restore) using [ncs-backup](../../resources/man/ncs-backup.1.md). -2. Check the currently active generation using the `/key-rotation/get-active-generation` action. -3. Re-encrypt all encrypted values with a new set of keys using the `/key-rotation/apply-new-key` action with the `new-key-generation` to rotate to as input.\ - The commit queue must be empty before running the action, or the action will fail, as the snapshot database is re-initialized. To wait for the commit queue to become empty, use the `wait-commit-queue` argument with the number of seconds to wait before failing. - -CLI example: - -``` -$ ${NCS_DIR}/bin/ncs-backup -$ ncs_cli -Cu admin -# key-rotation get-active-generation -active-generation -1 -# key-rotation apply-new-keys new-key-generation 0 wait-commit-queue 10 -result true -new-active-key-generation 0 -``` - -The data in CDB that is subject to re-encryption when executing the `/key-rotation/apply-new-key` action: - -* Encrypted types. -* Unions of encrypted types. -* Service metadata (original attribute, reverse and forward diff set). -* NED secrets. -* Rollback files. -* History log. - -Under the hood, the`/key-rotation/apply-new-keys` action, when executed, performs the following steps: - -1. Starts an upgrade transaction that will be used when re-encrypting the datastore. -2. Load the new active cryptographic keys into CDB and persist them. -3. Sync HA. -4. Re-encrypt data. -5. Drops the CDB snapshot database. -6. Commits data. -7. Restart NSO VMs. -8. End upgrade. - -## Reloading After Changes to the Cryptographic Keys - -1. Before changing the cryptographic keys, always [take a backup](../management/system-management/#backup-and-restore) using [ncs-backup](../../resources/man/ncs-backup.1.md). Also, back up the external key file, default `${NCS_CONFIG_DIR}/ncs.crypto_keys`, or the `${NCS_CONFIG_DIR}/ncs.conf` file, depending on where the keys are stored. -2. Suppose you have previously provided keys in the legacy format and wish to switch to `/ncs-config/encrypted-strings/key-rotation` or `external-keys` with the initial line `EXTERNAL_KEY_FORMAT=2`. In that case, you must provide the currently used keys as generation `-1`. The new keys can have any non-negative generation number. -3. Replace the external key file or `ncs.conf` file depending on where the keys are stored. -4. Issue `ncs --reload` to reload the cryptographic keys. -5. Ensure commit queues are empty or wait for them to become empty. -6. Execute`/key-rotation/apply-new-keys` action to change the active generation, for example, from `-1` to `new-key-generation 0` as shown in the CLI example above. - -{% hint style="info" %} -In a high-availability setting, keys must be identical on all nodes before attempting key rotation. Otherwise, the action will abort. The node executing the action will initiate the key reload for all nodes. -{% endhint %} - -## Migrating 3DES Encrypted Values - -NSO 6.5 removed support for 3DES encryption since the algorithm is no longer deemed sufficiently secure. If you are migrating from an older version and you have data using the `tailf:des3-cbc-encrypted-string` YANG type, NSO will no longer be able to read this data. In fact, compiling a YANG module using this type will produce an error. - -To avoid losing data when upgrading to NSO 6.5 or later, you must first update all the YANG data models and change the `tailf:des3-cbc-encrypted-string` type to either `tailf:aes-cfb-128-encrypted-string` or `tailf:aes-256-cfb-128-encrypted-string`. Compile the updated models and then perform a package upgrade for the affected packages. - -While upgrading the packages, the automatic CDB schema upgrade will re-encrypt the data in the new (AES) format. At this point you are ready to upgrade to the new NSO version that no longer supports 3DES. diff --git a/administration/advanced-topics/ipc-connection.md b/administration/advanced-topics/ipc-connection.md deleted file mode 100644 index 69eec231..00000000 --- a/administration/advanced-topics/ipc-connection.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -description: Connect client libraries to NSO with IPC. ---- - -# IPC Connection - -Client libraries connect to NSO for inter-process communication (IPC) using TCP or Unix domain sockets. - -If NSO is configured to use TCP sockets for IPC, you can tell NSO which address to use for these connections through the `/ncs-config/ncs-ipc-address/ip` (default value 127.0.0.1) and `/ncs-config/ncs-ipc-address/port` (default value 4569) elements in `ncs.conf`. If you change these values, you will likely need to configure the clients accordingly. Note that these values have security implications; see [Security Issues](../installation-and-deployment/deployment/secure-deployment.md#securing-ipc-access). In particular, changing the address away from 127.0.0.1 may allow unauthenticated remote connections. - -Many of the clients read the environment variables `NCS_IPC_ADDR` and `NCS_IPC_PORT` to determine if something other than the default is to be used, but others might need source code changes. This is a list of clients that communicate with NSO and what needs to be done when `ncs-ipc-address` is changed. - -
ClientChanges required
Remote commands via the ncs commandRemote commands, such as ncs --reload, check the environment variables NCS_IPC_ADDR and NCS_IPC_PORT.
CLI toolsThe Command Line Interface (CLI) client ncs_cli and similar commands, such as ncs_cmd and ncs_load, check the environment variables NCS_IPC_ADDR and NCS_IPC_PORT. Alternatively, many of them also support command-line options.
CDB and MAAPI clientsThe address supplied to Cdb.connect() and Maapi.connect() must be changed.
Data provider API clientsThe address supplied to Dp constructor socket must be changed.
Notification API clientsThe new address must be supplied to the socket for the Nofif constructor.
- -Likewise, if NSO is configured to use Unix domain sockets for IPC and you have changed the path under `/ncs-config/ncs-local-ipc/path` in `ncs.conf`, you can tell clients to use the new path through the `NCS_IPC_PATH` environment variable. Clients must also have filesystem permission to access the IPC path, or they will not be able to communicate with the NSO daemon process. - -To run more than one instance of NSO on the same host (which can be useful in development scenarios), each instance needs its own IPC socket. If using TCP for IPC, set `/ncs-config/ncs-ipc-address/port` in `ncs.conf` to different values for each instance. If, instead, you are using Unix sockets for IPC, set `/ncs-config/ncs-local-ipc/path` in `ncs.conf` to different values. In either case, you may also need to change the NETCONF and CLI over SSH ports under `/ncs-config/netconf/transport` and `/ncs-config/cli/ssh` by either disabling them or changing their values. - -## Restricting Access to the IPC Socket - -By default, clients connecting to the IPC socket are considered trusted, i.e., there is no authentication required, as the system relies on the use of 127.0.0.1 for `/ncs-config/ncs-ipc-address/ip` or Unix domain sockets to prevent remote access. In case this is not sufficient, such as when untrusted users have shell access on the system where NSO runs, it is possible to further restrict the access to the IPC socket. - -If Unix domain sockets are used, you can leverage Unix filesystem permissions for the socket path to limit which OS users and groups can initiate connections to the socket. NSO may also perform additional authentication of the connecting users; see [Authenticating IPC Access](../management/aaa-infrastructure.md#authenticating-ipc-access). - -For TCP sockets, you can enable an access check by setting the `ncs.conf` element `/ncs-config/ncs-ipc-access-check/enabled` to `true`, and specifying a filename for `/ncs-config/ncs-ipc-access-check/filename`. The file should contain a shared secret, i.e., a random (printable ASCII) character string. Clients connecting to the IPC socket will then be required to prove that they have knowledge of the secret through a challenge handshake before they are allowed access to the NSO functions provided via the IPC socket. - -{% hint style="info" %} -The access permissions on this file must be restricted via OS file permissions, such that it can only be read by the NSO daemon and client processes that are allowed to connect to the IPC port. E.g. if both the daemon and the clients run as root, the file can be owned by root and have only "read by owner" permission (i.e. mode 0400). Another possibility is to have a group that only the daemon and the clients belong to, set the group ID of the file to that group, and have only "read by group" permission (i.e. mode 040). -{% endhint %} - -To provide the secret to the client libraries and inform them that they need to use the access check handshake, you have to set the environment variable `NCS_IPC_ACCESS_FILE` to the full pathname of the file containing the secret. This is sufficient for all the clients mentioned above, i.e., there is no need to change the application code to support or enable this check. - -{% hint style="info" %} -The access check must be either enabled or disabled for both the daemon and the clients. E.g., if `/ncs-config/ncs-ipc-access-check/enabled` in `ncs.conf` is not set to `true` but clients are started with the environment variable `NCS_IPC_ACCESS_FILE` pointing to a file with a secret, the client connections will fail. -{% endhint %} diff --git a/administration/advanced-topics/ipv6-on-northbound-interfaces.md b/administration/advanced-topics/ipv6-on-northbound-interfaces.md deleted file mode 100644 index c589bb0a..00000000 --- a/administration/advanced-topics/ipv6-on-northbound-interfaces.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: Learn about using IPv6 on NSO's northbound interfaces. ---- - -# IPv6 on Northbound Interfaces - -NSO supports access to all northbound interfaces via IPv6, and in the most simple case, i.e., IPv6-only access, this is just a matter of configuring an IPv6 address (typically the wildcard address `::`) instead of IPv4 for the respective agents and transports in `ncs.conf`, e.g., `/ncs-config/cli/ssh/ip` for SSH connections to the CLI or `/ncs-config/netconf-north-bound/transport/ssh/ip` for SSH to the NETCONF agent. The SNMP agent configuration is configured via one of the other northbound interfaces rather than via `ncs.conf`, see [NSO SNMP Agent](../../development/core-concepts/northbound-apis/#the-nso-snmp-agent) in Northbound APIs. For example, via the CLI, we would set `snmp agent ip` to the desired address. All these addresses default to the IPv4 wildcard address `0.0.0.0`. - -In most IPv6 deployments, it will, however, be necessary to support IPv6 and IPv4 access simultaneously. This requires that both IPv4 and IPv6 addresses are configured, typically `0.0.0.0` plus `::`. To support this, there is in addition to the `ip` and `port` leafs also a list `extra-listen` for each agent and transport, where additional IP addresses and port pairs can be configured. Thus, to configure the CLI to accept SSH connections to port 2024 on any local IPv6 address, in addition to the default (port 2024 on any local IPv4 address), we can add an `` section under `/ncs-config/cli/ssh` in `ncs.conf`: - -```xml - - true - - - - true - 0.0.0.0 - 2024 - - - :: - 2024 - - - - - ... - -``` - -To configure the SNMP agent to accept requests to port 161 on any local IPv6 address, we could similarly use the CLI and give the command: - -```bash -admin@ncs(config)# snmp agent extra-listen :: 161 -``` - -The `extra-listen` list can take any number of address/port pairs; thus, this method can also be used when we want to accept connections/requests on several specified (IPv4 and/or IPv6) addresses instead of the wildcard address or when we want to use multiple ports. diff --git a/administration/advanced-topics/layered-service-architecture.md b/administration/advanced-topics/layered-service-architecture.md deleted file mode 100644 index 251c27e9..00000000 --- a/administration/advanced-topics/layered-service-architecture.md +++ /dev/null @@ -1,1217 +0,0 @@ ---- -description: Design large and scalable NSO applications using LSA. ---- - -# Layered Service Architecture - -Layered Service Architecture (LSA) is a design approach for massively large and scalable NSO applications. Large service providers and enterprises can use it to manage services for millions of users, ranging over several hundred thousand managed devices. Such scale requires special consideration since a single NSO instance no longer suffices and LSA helps you address this challenge. - -## Going Big - -At some point, scaling up hits the law of diminishing returns. Effectively, adding more resources to the NSO server becomes prohibitively expensive. To further increase the throughput of the whole system, you can share the load across multiple instances, in a scale-out fashion. - -You achieve this by splitting a service into a main, upper-layer part, and one or more lower-layer parts. The upper part controls and dispatches work to the lower parts. This is the same approach as using a customer-facing service (CFS) and a resource-facing service (RFS). However, here the CFS code (the upper-layer part) runs in a different NSO node than the RFS code (the lower-layer parts). What is more, the lower-layer parts can be spread across multiple NSO nodes. - -Each RFS node is responsible for its own set of managed devices, mounted under its `/devices` tree, and the upper-layer, CFS node only concerns itself with the RFS nodes. So, the CFS node only mounts the RFS nodes under its `/devices` tree, not managed devices directly. The main advantage of this architecture is that you can add many device RFS nodes that collectively manage a huge number of actual devices—much more than a single node could. - -

Layered CFS/RFS architecture

- -## Is LSA for Me? - -While it is tempting to design the system in the most scalable way from the start, it comes with a cost. Compared to a single, non-LSA setup, the automation system now becomes distributed across multiple nodes, with all the complexity that entails. For example, in a non-distributed system, the communication between different parts has mostly negligible latency and hardly ever fails. That is certainly not true anymore for distributed systems as we know them today, including LSA. - -More practically, taking a service in NSO and deploying a single instance on an LSA system is likely to take longer and have a higher chance of failure compared to a non-LSA system, because additional network communication is involved. - -Moreover, multiple NSO nodes present a higher operational complexity and administrative burden. There is no longer a “single pane of glass” view of all the individual devices. That's why you must weigh the benefits of the LSA approach against the scale at which you operate. When LSA starts making sense will depend on the type of devices you manage, the services you have, the geographical distribution of resources, and so on. - -A distributed system can push the overall throughput way beyond what a single instance can do. But you will achieve a much better outcome by first focusing on eliminating the bottlenecks in the provisioning code, as discussed in [Scaling and Performance Optimization](../../development/advanced-development/scaling-and-performance-optimization.md). Only when that proves insufficient, consider deploying LSA. - -LSA also addresses the memory limitations of NSO when device configurations become very large (individually or all together). If the NSO server is memory-constrained and more memory cannot be added, the LSA approach can be a solution. - -Another challenge that LSA may help you overcome is scaling organizationally. When many teams share the same NSO instance, it can get hard to separate the different concerns and responsibilities. Teams may also have different cadences or preferences for upgrades, resulting in friction. With LSA, it becomes possible to create a clearer separation. The CFS node and the RFS nodes can have different release cycles (as long as the YANG upgrade rules are followed) and each can be upgraded independently. If a bug is found or a feature is missing in the RFS nodes, it can be fixed without affecting the CFS node, and vice versa. - -To summarize, the major advantage of this architecture is scalability. The solution scales horizontally, both at the upper and the lower layer, thus catering for truly massive deployments, but at the expense of the increased complexity. - -## Layered Service Design - -To take advantage of the scalability potential of LSA, your services must be designed in a layered fashion. Once the automation logic in NSO reaches a certain level of complexity, a stacked service design tends to emerge naturally. Often, you can extend it to LSA with relatively little change. The same is true for brand-new, green field designs. - -In other situations, you might need to invest some additional effort to split and orchestrate the work across multiple groups of devices. Examples are existing monolithic services or stacked service designs that require all RFSs to access all devices. - -### New, Greenfield Design - -If you are designing the service from scratch, you have the most freedom in choosing the partitioning of logic between CFS and RFS. The CFS must contain the YANG definition for the service and its configurable options that are available to the customer, perhaps through an order capture system north of the NSO. On the other hand, the RFS YANG models are internal to the service, that is, they are not used directly by the customer. So, you are free to design them in a way that makes the provisioning code as simple as possible. - -As an example, you might have a VLAN provisioning service where the CFS lets users select if the hosts on the VLAN can access the internet. Then you can divide provisioning into, let's say, an RFS service that configures the VLAN and the appropriate IP subnet across the data center switches, and another RFS service that configures the firewall to allow the traffic from the subnet to reach the internet. This design clearly separates the provisioned devices into two groups: firewalls and data center switches. Each group can be managed by a separate lower-layer NSO. - -### Existing Monolithic Application with Stacked Services - -Similar to a brand new design, an existing monolithic application that uses stacked services has already laid the groundwork for LSA-compatible design because of the existing division into two layers (upper and lower). - -A possible complication, in this case, is when each existing RFS touches all of the affected devices, and that makes it hard to partition devices across multiple lower-layer NSO nodes. For example, if one RFS manages the VLAN interface (the VLAN ID and layer 2 settings) and another RFS manages the IP configuration for this interface, that configuration very likely happens on the same devices. The solution in this situation could be to partition RFS services based on the data center that they operate in, such as one lower-layer NSO node for one data center, another lower-layer NSO for another data center, and so on. If that is not possible, an alternative is to redesign each RFS and split their responsibilities differently. - -#### Existing Monolithic Application - -The most complex, yet common case is when a single node NSO installation grows over time and you are faced with performance problems due to the new size. To leverage the LSA functionality, you must first split the service into upper- and lower-layer parts, which require a certain amount of effort. That is why the decision to use LSA should always be accompanied by a thorough analysis to determine what makes the system too slow. Sometimes, it is a result of a bad "must" expression in the service YANG code or similar. Fixing that is much easier than re-architecting the application. - -### Orchestrating the Work - -Regardless of whether you start with a green field design or extend an existing application, you must tackle the problem of dispatching the RFS instantiation to the correct lower-layer NSO node. - -Imagine a VPN application that uses a managed device on each site to securely connect to the private network. In a service provider network, this is usually done by the CPE. When a customer orders connectivity to an additional site (another leg of the VPN), the service needs to configure the site-local device (the CPE). As there will be potentially many such devices, each will be managed by one of the RFS nodes. However, the VPN service is managed centrally, through the CFS, which must: - -* Figure out which RFS node is responsible for the device for the new site (CPE). -* Dispatch the RFS instantiation to that particular RFS node, making sure the device is properly configured. - -NSO provides a mechanism to facilitate the second part, the actual dispatch, but the service logic must somehow select the correct RFS node. If the RFS nodes are geographically separated across different countries or different data centers, the CFS could simply infer or calculate the right RFS node based on service instance parameters, such as the physical location of the new site. - -A more flexible alternative is to use dynamic mapping. It can be as simple as a list of 2-tuples that map a device name to an RFS node, stored in the CDB. The trade-off is that the list must be maintained. It is straightforward to automate the maintenance of the list though, for example through NETCONF notifications whenever `/devices/device` on the RFS nodes is manipulated or by explicitly asking the CFS node to query the RFS nodes for their list of devices. - -Ultimately, the right approach to dispatch will depend on the complexity of your service and operational procedures. - -### Provisioning of an LSA Service Request - -Having designed a layered service with the CFS and RFS parts, the CFS must now communicate with the RFS that resides on a different node. You achieve that by adding the lower-layer (RFS) node as a managed device to the upper-layer (CFS) node. The CFS node must access the RFS data model on the lower-layer node, just like it accesses any other configuration on any managed device. But don't you need a NED to do this? Indeed, you do. That's why the RFS model needs to be specially compiled for the upper-layer node to use as part of NED and not a standalone service. A model compiled in this way is called a 'device compiled'. - -Let's then see how the LSA setup affects the whole service provisioning process. Suppose a new request arrives at the CFS node, such as a new service instance being created through RESTCONF by a customer order portal. The CFS runs the service mapping logic as usual; however, instead of configuring the network devices directly, the CFS configures the appropriate RFS nodes with the generated RFS service instance data. This is the dispatch logic in action. - -

LSA Request Flow

- -As the configuration for the lower-layer nodes happens under the `/devices/device` tree, it is picked up and pushed to the relevant NSO instances by the NED. The NED sends the appropriate NETCONF edit-config RPCs, which trigger the RFS FASTMAP code at the RFS nodes. The RFS mapping logic constructs the necessary network configuration for each RFS instance and the RFS nodes update the actual network devices. - -In case the commit queue feature is not being used, this entire sequence is serialized through the system as a whole. It means that if another northbound request arrives at the CFS node while the first request is being processed, the second request is synchronously queued at the CFS node, waiting for the currently running transaction to either succeed or fail. - -If the code on the RFS nodes is reactive, it will likely return without much waiting, since the RFM applications are usually very fast during their first round of execution. But that will still have a lower performance than using the commit queue since the execution is serialized eventually when modifying devices. To maximize throughput, you also need to enable the commit queue functionality throughout the system. - -### Implementation Considerations - -The main benefit of LSA is that it scales horizontally at the RFS node layer. If one RFS node starts to become overloaded, it's easy to bring up an additional one, to share the load. Thus LSA caters to scalability at the level of the number of managed devices. However, each RFS node needs to host all the RFSs that touch the devices it manages under its `/devices/device` tree. There is still one, and only one, NSO node that directly manages a single device. - -Dividing a provisioning application into upper and lower-layer services also increases the complexity of the application itself. For example, to follow the execution of a reactive or nano RFS, typically an additional NETCONF notification code must be written. The notifications have to be sent from the RFS nodes and received and processed by the CFS code. This way, if something goes wrong at the device layer, the information is relayed all the way to the top level of the system. - -Furthermore, it is highly recommended that LSA applications enable the commit queue on all NSO nodes. If the commit queue is not enabled, the slowest device on the network will limit the overall throughput, significantly reducing the benefits of LSA. - -Finally, if the two-layer approach proves to be insufficient due to requirements at the CFS node, you can extend it to three layers, with an additional layer of NSO nodes between the CFS and RFS layers. - -## LSA Examples - -### Greenfield LSA Application - -This section describes a small LSA application, which exists as a running example in the [examples.ncs/layered-services-architecture/lsa-single-version-deployment](https://github.com/NSO-developer/nso-examples/tree/6.6/layered-services-architecture/lsa-single-version-deployment) directory. - -The application is a slight variation on the [examples.ncs/service-management/rfs-service](https://github.com/NSO-developer/nso-examples/tree/6.6/service-management/rfs-service) example where the YANG code has been split up into an upper-layer and a lower-layer implementation. The example topology (based on netsim for the managed devices, and NSO for the upper/lower layer NSO instances) looks like the following: - -

Example LSA architecture

- -The upper layer of the YANG service data for this example looks like the following: - -```yang -module cfs-vlan { - ... - list cfs-vlan { - key name; - leaf name { - type string; - } - - uses ncs:service-data; - ncs:servicepoint cfs-vlan; - - leaf a-router { - type leafref { - path "/dispatch-map/router"; - } - mandatory true; - } - leaf z-router { - type leafref { - path "/dispatch-map/router"; - } - mandatory true; - } - leaf iface { - type string; - mandatory true; - } - leaf unit { - type int32; - mandatory true; - } - leaf vid { - type uint16; - mandatory true; - } - } -} -``` - -Instantiating one CFS we have: - -``` -admin@upper-nso% show cfs-vlan -cfs-vlan v1 { - a-router ex0; - z-router ex5; - iface eth3; - unit 3; - vid 77; -} -``` - -The provisioning code for this CFS has to make a decision on where to instantiate what. In this example the "what" is trivial, it's the accompanying RFS, whereas the "where" is more involved. The two underlying RFS nodes, each manage 3 netsim routers, thus given the input, the CFS code must be able to determine which RFS node to choose. In this example, we have chosen to have an explicit map, thus on the `upper-nso` we also have: - -``` -admin@upper-nso% show dispatch-map -dispatch-map ex0 { - rfs-node lower-nso-1; -} -dispatch-map ex1 { - rfs-node lower-nso-1; -} -dispatch-map ex2 { - rfs-node lower-nso-1; -} -dispatch-map ex3 { - rfs-node lower-nso-2; -} -dispatch-map ex4 { - rfs-node lower-nso-2; -} -dispatch-map ex5 { - rfs-node lower-nso-2; -} -``` - -So, we have a template CFS code that does the dispatching to the right RFS node. - -```xml - - - - - - - {string(deref(current())/../rfs-node)} - - - - {string(/name)} - - {current()} - {/iface} - {/unit} - {/vid} - Interface owned by CFS: {/name} - - - - - - -``` - -This technique for dispatching is simple and easy to understand. The dispatching might be more complex, it might even be determined at execution time dependent on CPU load. It might be (as in this example) inferred from input parameters or it might be computed. - -The result of the template-based service is to instantiate the RFS, at the RFS nodes. - -First, let's have a look at what happened in the upper-nso. Look at the modifications but ignore the fact that this is an LSA service: - -``` -admin@upper-nso% request cfs-vlan v1 get-modifications no-lsa -cli { - local-node { - data devices { - device lower-nso-1 { - config { - + rfs-vlan:vlan v1 { - + router ex0; - + iface eth3; - + unit 3; - + vid 77; - + description "Interface owned by CFS: v1"; - + } - } - } - device lower-nso-2 { - config { - + rfs-vlan:vlan v1 { - + router ex5; - + iface eth3; - + unit 3; - + vid 77; - + description "Interface owned by CFS: v1"; - + } - } - } - } - } -} -``` - -Just the dispatched data is shown. As `ex0` and `ex5` reside on different nodes, the service instance data has to be sent to both `lower-nso-1` and `lower-nso-2`. - -Now let's see what happened in the `lower-nso`. Look at the modifications and take into account that these are LSA nodes (this is the default): - -``` -admin@upper-nso% request cfs-vlan v1 get-modifications -cli { - local-node { - ..... - } - lsa-service { - service-id /devices/device[name='lower-nso-1']/config/rfs-vlan:vlan[name='v1'] - data devices { - device ex0 { - config { - r:sys { - interfaces { - + interface eth3 { - + enabled; - + unit 3 { - + enabled; - + description "Interface owned by CFS: v1"; - + vlan-id 77; - + } - + } - } - } - } - } - } - } - lsa-service { - service-id /devices/device[name='lower-nso-2']/config/rfs-vlan:vlan[name='v1'] - data devices { - device ex5 { - config { - r:sys { - interfaces { - + interface eth3 { - + enabled; - + unit 3 { - + enabled; - + description "Interface owned by CFS: v1"; - + vlan-id 77; - + } - + } - } - } - } - } - } - } -``` - -Both the dispatched data and the modification of the remote service are shown. As `ex0` and `ex5` reside on different nodes, the service modifications of the service `rfs-vlan` on both `lower-nso-1` and `lower-nso-2` are shown. - -The communication between the NSO nodes is of course NETCONF. - -``` -admin@upper-nso% set cfs-vlan v1 a-router ex0 z-router ex5 iface eth3 unit 3 vid 78 -[ok][2016-10-20 16:52:45] - -[edit] -admin@upper-nso% commit dry-run outformat native -native { - device { - name lower-nso-1 - data - - - - - test-then-set - rollback-on-error - - - - v1 - 78 - - -1 - - - - - - } - ........... - .... -``` - -The YANG model at the lower layer, also known as the RFS layer, is similar to the CFS, but slightly different: - -```yang -module rfs-vlan { - - ... - - list vlan { - key name; - leaf name { - tailf:cli-allow-range; - type string; - } - - uses ncs:service-data; - ncs:servicepoint "rfs-vlan"; - - leaf router { - type string; - } - leaf iface { - type string; - mandatory true; - } - leaf unit { - type int32; - mandatory true; - } - leaf vid { - type uint16; - mandatory true; - } - leaf description { - type string; - mandatory true; - } - } -} -``` - -The task for the RFS provisioning code here is to actually provision the designated router. If we log into one of the lower layer NSO nodes, we can check the following. - -``` -admin@lower-nso-1> show configuration vlan -vlan v1 { - router ex0; - iface eth3; - unit 3; - vid 77; - description "Interface owned by CFS: v1"; -} -[ok][2016-10-20 17:01:08] -admin@lower-nso-1> request vlan v1 get-modifications -cli { - local-node { - data devices { - device ex0 { - config { - r:sys { - interfaces { - + interface eth3 { - + enabled; - + unit 3 { - + enabled; - + description "Interface owned by CFS: v1"; - + vlan-id 77; - + } - + } - } - } - } - } - } - } -} -``` - -To conclude this section, the final remark here is that to design a good LSA application, the trick is to identify a good layering for the service data models. The upper layer, the CFS layer is what is exposed northbound, and thus requires a model that is as forward-looking as possible since that model is what a system north of NSO integrates to, whereas the lower layer models, the RFS models can be viewed as "internal system models" and they can be more easily changed. - -### Greenfield LSA Application Designed for Easy Scaling - -In this section, we'll describe a lightly modified version of the example in the previous section. The application we describe here exists as a running example under [examples.ncs/layered-services-architecture/lsa-scaling](https://github.com/NSO-developer/nso-examples/tree/6.6/layered-services-architecture/lsa-scaling). - -Sometimes it is desirable to be able to easily move devices from one lower LSA node to another. This makes it possible to easily expand or shrink the number of lower LSA nodes. Additionally, it is sometimes desirable to avoid HA pairs for replication but instead use a common store for all lower LSA devices, such as a distributed database, or a common file system. - -The above is possible provided that the LSA application is structured in certain ways. - -* The lower LSA nodes only expose services that manipulate the configuration of a single device. We call these devices RFSs, or dRFS for short. -* All services are located in a way that makes it easy to extract them, for example in /drfs:dRFS/device - - ```yang - container dRFS { - list device { - key name; - leaf name { - type string; - } - } - } - ``` -* No RFS takes place on the lower LSA nodes. This avoids the complication with locking and distributed event handling. -* The LSA nodes need to be set up with the proper NEDs and with auth groups such that a device can be moved without having to install new NEDs or update auth groups. - -Provided that the above requirements are met, it is possible to move a device from one lower LSA node by extracting the configuration from the source node and installing it on the target node. This, of course, requires that the source node is still alive, which is normally the case when HA-pairs are used. - -An alternative to using HA-pairs for the lower LSA nodes is to extract the device configuration after each modification to the device and store it in some central storage. This would not be recommended when high throughput is required but may make sense in certain cases. - -In the example application, there are two packages on the lower LSA nodes that provide this functionality. The package `inventory-updater` installs a database subscriber that is invoked every time any device configuration is modified, both in the preparation phase and in the commit phase of any such transaction. It extracts the device and dRFS configuration, including service metadata, during the preparation phase. If the transaction proceeds to a full commit, the package is again invoked and the extracted configuration is stored in a file in the directory `db_store`. - -The other package is called `device-actions`. It provides three actions: `extract-device`, `install-device`, and `delete-device`. They are intended to be used by the upper LSA node when moving a device either from a lower LSA node or from `db_store`. - -In the upper LSA node, there is one package for coordinating the movement, called `move-device`. It provides an action for moving a device from one lower LSA node to another. For example when invoked to move device `ex0` from `lower-1` to `lower-2` using the action - -```cli -request move-device move src-nso lower-1 dest-nso lower-2 device-name ex0 -``` - -it goes through the following steps: - -* A partial lock is acquired on the upper-nso for the path `/devices/device[name=lower-1]/config/dRFS/device[name=ex0]` to avoid any changes to the device while the device is in the process of being moved. -* The device and dRFS configuration are extracted in one of two ways: - - * Read the configuration from `lower-1` using the action - - ```cli - request device-action extract-device name ex0 - ``` - * Read the configuration from some central store, in our case the file system in the directory. `db_store`. - - The configuration will look something like this - - ``` - devices { - device ex0 { - address 127.0.0.1; - port 12022; - ssh { - ... - /* Refcount: 1 */ - /* Backpointer: [ /drfs:dRFS/drfs:device[drfs:name='ex0']/rfs-vlan:vlan[rfs-vlan:name='v1'] ] */ - interface eth3 { - ... - } - ... - } - } - dRFS { - device ex0 { - vlan v1 { - private { - ... - } - } - } - } - ``` -* Install the configuration on the `lower-2` node. This can be done by running the action: - - ```cli - request device-action install-device name ex0 config - ``` - - This will load the configuration and commit using the flags `no-deploy` and `no-networking`. -* Delete the device from `lower-1` by running the action - - ```cli - request device-action delete-device name ex0 - ``` -* Update mapping table - - ``` - dispatch-map ex0 { - rfs-node lower-nso-2; - } - ``` -* Release the partial lock for `/devices/device[name=lower-1]/config/dRFS/device[name=ex0]`. -* Re-deploy all services that have touched the device. The services all have backpointers from `/devices/device{lower-1}/config/dRFS/device{ex0}`. They are `re-deployed` using the flags `no-lsa` and `no-networking`. -* Finally, the action runs `compare-config` on `lower-1` and `lower-2`. - -With this infrastructure in place, it is fairly straightforward to implement actions for re-balancing devices among lower LSA nodes, as well as evacuating all devices from a given lower LSA node. The example contains implementations of those actions as well. - -### Re-architecting an Existing VPN Application for LSA - -If we do not have the luxury of designing our NSO service application from scratch, but rather are faced with extending/changing an existing, already deployed application into the LSA architecture we can use the techniques described in this section. - -Usually, the reasons for re-architecting an existing application are performance-related. - -In the NSO example collection, two popular examples are the [examples.ncs/service-management/mpls-vpn-java](https://github.com/NSO-developer/nso-examples/tree/6.6/service-management/mpls-vpn-java) and [examples.ncs/service-management/mpls-vpn-python](https://github.com/NSO-developer/nso-examples/tree/6.6/service-management/mpls-vpn-python) examples. Those example contains an almost "real" VPN provisioning example whereby VPNs are provisioned in a network of CPEs, PEs, and P routers according to this picture: - -

VPN network

- -The service model in this example roughly looks like this: - -```yang - list l3vpn { - description "Layer3 VPN"; - - key name; - leaf name { - type string; - } - - leaf route-distinguisher { - description "Route distinguisher/target identifier unique for the VPN"; - mandatory true; - type uint32; - } - - list endpoint { - key "id"; - leaf id { - type string; - } - leaf ce-device { - mandatory true; - type leafref { - path "/ncs:devices/ncs:device/ncs:name"; - } - } - - leaf ce-interface { - mandatory true; - type string; - } - - .... - - leaf as-number { - tailf:info "CE Router as-number"; - type uint32; - } - } - container qos { - leaf qos-policy { - ...... -``` - -There are several interesting observations on this model code related to the Layered Service Architecture. - -* Each instantiated service has a list of endpoints and CPE routers. These are modeled as a leafref into the /devices tree. This has to be changed if we wish to change this application into an LSA application since the /devices tree at the upper layer doesn't contain the actual managed routers. Instead, the /devices tree contains the lower layer RFS nodes. -* There is no connectivity/topology information in the service model. Instead, the `mpls-vpn` example has topology information on the side, and that data is used by the provisioning code. That topology information for example contains data on which CE routers are directly connected to which PE router. - - Remember from the previous section, that one of the additional complications of an LSA application is the dispatching part. The dispatching problem fits well into the pattern where we have topology information stored on the side and let the provisioning FASTMAP code use that data to guide the provisioning. One straightforward way would be to augment the topology information with additional data, indicating which RFS node is used to manage a specific managed device. - -By far the easiest way to change an existing monolithic NSO application into the LSA architecture is to keep the service model at the upper layer and lower layer almost identical, only changing things like leafrefs directly into the /devices tree which obviously breaks. - -In this example, the topology information is stored in a separate container `share-data` and propagated to the LSA nodes by means of service code. - -The example [examples.ncs/layered-services-architecture/mpls-vpn-lsa](https://github.com/NSO-developer/nso-examples/tree/6.6/layered-services-architecture/mpls-vpn-lsa) example does exactly this, the upper layer data model in `upper-nso/packages/l3vpn/src/yang/l3vpn.yang` now looks as: - -```yang - list l3vpn { - description "Layer3 VPN"; - - key name; - leaf name { - type string; - } - - leaf route-distinguisher { - description "Route distinguisher/target identifier unique for the VPN"; - mandatory true; - type uint32; - } - - list endpoint { - key "id"; - leaf id { - type string; - } - leaf ce-device { - mandatory true; - type string; - } - ....... -``` - -The `ce-device` leaf is now just a regular string, not a leafref. - -So, instead of an NSO topology that looks like: - -

NSO topology

- -\ -We want an NSO architecture that looks like this: - -

NSO LSA topology

- -The task for the upper layer FastMap code is then to instantiate a copy of itself on the right lower layer NSO nodes. The upper layer FastMap code must: - -* Determine which routers, (CE, PE, or P) will be touched by its execution. -* Look in its dispatch table, which lower-layer NSO nodes are used to host these routers. -* Instantiate a copy of itself on those lower layer NSO nodes. One extremely efficient way to do that is to use the `Maapi.copyTree()` method. The code in the example contains code that looks like this: - - ```java - public Properties create( - .... - NavuContainer lowerLayerNSO = .... - - Maapi maapi = service.context().getMaapi(); - int tHandle = service.context().getMaapiHandle(); - NavuNode dstVpn = lowerLayerNSO.container("config"). - container("l3vpn", "vpn"). - list("l3vpn"). - sharedCreate(serviceName); - ConfPath dst = dstVpn.getConfPath(); - ConfPath src = service.getConfPath(); - - maapi.copyTree(tHandle, true, src, dst); - ``` - -Finally, we must make a minor modification to the lower layer (RFS) provisioning code too. Originally, the FastMap code wrote all config for all routers participating in the VPN, now with the LSA partitioning, each lower layer NSO node is only responsible for the portion of the VPN that involves devices that reside in its /devices tree, thus the provisioning code must be changed to ignore devices that do not reside in the /devices tree. - -### Re-architecting Details - -In addition to conceptual changes of splitting into upper- and lower-layer parts, migrating an existing monolithic application to LSA may also impact the models used. In the new design, the upper-layer node contains the (more or less original) CFS model as well as the device-compiled RFS model, which it requires for communication with the RFS nodes. In a typical scenario, these are two separate models. So, for example, they must each use a unique namespace. - -To illustrate the different YANG files and namespaces used, the following text describes the process of splitting up an example monolithic service. Let's assume that the original service resides in a file, `myserv.yang`, and looks like the following: - -```yang -module myserv { - - namespace "http://example.com/myserv"; - prefix ms; - - ..... - - list srv { - key name; - leaf name { - type string; - } - - uses ncs:service-data; - ncs:servicepoint vlanspnt; - - leaf router { - type leafref { - path "/ncs:devices/ncs:device/ncs:name"; - ..... - } -} -``` - -In an LSA setting, we want to keep this module as close to the original as possible. We clearly want to keep the namespace, the prefix, and the structure of the YANG identical to the original. This is to not disturb any provisioning systems north of the original NSO. Thus with only minor modifications, we want to run this module at the CFS node, but with non-applicable leafrefs removed, thus at the CFS node we would get: - -```yang -module myserv { - - namespace "http://example.com/myserv"; - prefix ms; - - ..... - - list srv { - key name; - leaf name { - type string; - } - - uses ncs:service-data; - ncs:servicepoint vlanspnt; - - leaf router { - type string; - ..... - } -} -``` - -Now, we want to run almost the same YANG module at the RFS node, however, the namespace must be changed. For the sake of the CFS node, we're going to NED compile the RFS and NSO doesn't like the same namespace to occur twice, thus for the RFS node, we would get a YANG module `myserv-rfs.yang` that looks like the following: - -```yang -module myserv-rfs { - - namespace "http://example.com/myserv-rfs"; - prefix ms-rfs; - - ..... - - list srv { - key name; - leaf name { - type string; - } - - uses ncs:service-data; - ncs:servicepoint vlanspnt; - - leaf router { - type leafref { - path "/ncs:devices/ncs:device/ncs:name"; - ..... - } -} -``` - -This file can, and should, keep the leafref as is. - -The final and last file we get is the compiled NED, which should be loaded in the CFS node. The NED is directly compiled from the RFS model, as an LSA NED. - -```bash -$ ncs-make-package --lsa-netconf-ned /path/to-rfs-yang myserv-rfs-ned -``` - -Thus, we end up with three distinct packages from the original one: - -1. The original, slated for the CFS node, with leafrefs removed. -2. The modified original, slated for the RFS node, with the namespace and the prefix changed. -3. The NED, compiled from the RFS node code, slated for the CFS node. - -## Deploying LSA - -The purpose of the upper CFS node is to manage all CFS services and to push the resulting service mappings to the RFS services. The lower RFS nodes are configured as devices in the device tree of the upper CFS node and the RFS services are created under the `/devices/device/config` accordingly. This is almost identical to the relation between a normal NSO node and the normal devices. However, there are differences when it comes to commit parameters and the commit queue, as well as some other LSA-specific features. - -Such a design allows you to decide whether you will run the same version of NSO on all nodes or not. Since some differences arise between the two options, this document distinguishes a single-version deployment from a multi-version one. - -Deployment of an LSA cluster where all the nodes have the same major version of NSO running is called a single version deployment. If the versions are different, then it is a multi-version deployment, since the packages on the CFS node must be managed differently. - -The choice between the two deployment options depends on your functional needs. The single version is easier to maintain and is a good starting point but is less flexible. While it is possible to migrate from one to the other, the migration from a single version to a multi-version is typically easier than the other way around. Still, every migration requires some effort, so it is best to pick one approach and stick to it. - -You can find working examples of both deployment types in the [examples.ncs/layered-services-architecture/lsa-single-version-deployment](https://github.com/NSO-developer/nso-examples/tree/6.6/layered-services-architecture/lsa-single-version-deployment) and [examples.ncs/layered-services-architecture/lsa-multi-version-deployment](https://github.com/NSO-developer/nso-examples/tree/6.6/layered-services-architecture/lsa-multi-version-deployment) folders, respectively. - -### RFS Nodes Setup - -The type of deployment does not affect the RFS nodes. In general, the RFS nodes act very much like ordinary standalone NSO instances but only support the RFS services. - -Configure and set up the lower RFS nodes as you would a standalone node, by making sure the necessary NED and RFS packages are loaded and the managed network devices added. This requires you to have already decided on the distribution of devices to lower RFS nodes. The RFS packages are ordinary service packages. - -The only LSA-specific requirement is that these nodes enable NETCONF communication northbound, as this is how the upper CFS node will interact with them. To enable NETCONF northbound, ensure that a configuration similar to the following is present in the `ncs.conf` of every RFS node: - -```xml - - true - - - true - 0.0.0.0 - 2022 - - - -``` - -One thing to note is that you do not need to explicitly enable the commit queue on the RFS nodes, even if you intend to use LSA with the commit queue feature. The upper CFS node is aware of the LSA setup and will propagate the relevant commit flags to the lower RFS nodes automatically. - -If you wish to enable the commit queue by default, that is, even for transactions originating on the RFS node (non-LSA), you are strongly encouraged to enable it globally, through the `/devices/global-settings/commit-queue/enabled-by-default` setting on all the RFS nodes and, importantly, the upper CFS node too. Otherwise, you may end up in a situation where only a part of the transaction runs through the commit queue. In that case, the `rollback-on-error` commit queue error option will not work correctly, as it can't roll back the full original transaction but just the part that went through the commit queue. This can result in an inconsistent network state. - -### CFS Node Setup - -Regardless of single or multi-version deployment, the upper CFS node has the lower RFS nodes configured as devices under the `/devices/device` tree. The CFS node communicates with these devices through NETCONF and must have the correct `ned-id` configured for each lower RFS node. The `ned-id` is set under `/devices/device/device-type/netconf/ned-id`, as for any NETCONF device. - -The part that is specific to LSA is the actual `ned-id` used. This has to be `ned:lsa-netconf` or a `ned-id` derived from it. What is more, the `ned-id` depends on the deployment type. For a single-version deployment, you can use the `lsa-netconf` value directly. This `ned-id` is built-in (defined in `tailf-ncs-ned.yang`) and available in NSO without any additional packages. - -So the configuration for the RFS device in the CFS node would look similar to: - -```cli -admin@upper-nso% show devices device | display-level 4 -device lower-nso-1 { - lsa-remote-node lower-nso-1; - authgroup default; - device-type { - netconf { - ned-id lsa-netconf; - } - } - state { - admin-state unlocked; - } -} -``` - -Notice the use of the `lsa-remote-node` instead of the `address` (and `port`) as is usually done. This setting identifies the device as a lower-layer LSA node and instructs NSO to use connection information provided under `cluster` configuration. - -The value of `lsa-remote-node` references a `cluster remote-node`, such as the following: - -```cli -admin@upper-nso% show cluster remote-node -remote-node lower-nso-1 { - address 127.0.2.1; - authgroup default; -} -``` - -In addition to `devices device`, the `authgroup` value is again required here and refers to `cluster authgroup`, not the device one. Both authgroups must be configured correctly for LSA to function. - -Having added device and cluster configuration for all RFS nodes, you should update the SSH host keys for both, the `/devices/device` and `/cluster/remote-node` paths. For example: - -```cli -admin@upper-nso% request devices device lower-nso-* ssh fetch-host-keys -admin@upper-nso% request cluster remote-node lower-nso-* ssh fetch-host-keys -``` - -Moreover, the RFS NSO nodes have an extra configuration that may not be visible to the CFS node, resulting in out-of-sync behavior. You are strongly encouraged to set the `out-of-sync-commit-behaviour` value to `accept`, with a command such as: - -```cli -admin@upper-nso% set devices device lower-nso-* out-of-sync-commit-behaviour accept -``` - -At the same time you should also enable the `/cluster/device-notifications`, which will allow the CFS node to receive the forwarded device notifications from the RFS nodes, and `/cluster/commit-queue`, to enable the commit queue support for LSA. Without the latter, you will not be able to use the `commit commit-queue async` command, for example. - -If you wish to enable the commit queue by default, you should do so by setting the `/devices/global-settings/commit-queue/enabled-by-default` on the CFS node. Do not use per device or per device group configuration, for the same reason you should avoid it on the RFS nodes. - -#### Multi-Version Deployment - -If you plan a single-version deployment, the preceding steps are sufficient. For a multi-version deployment, on the other hand, there are two additional tasks to perform. - -First, you will need to install the correct Cisco-NSO LSA NED package (or packages if you need to support more versions). Each NSO release includes these packages that are specifically tailored for LSA. They are used by the upper CFS node if the lower RFS nodes are running a different version than the CFS node itself. The packages are named `cisco-nso-nc-X.Y` where X.Y are the two most significant numbers of the NSO release (the major version) that the package supports. So, if your RFS nodes are running NSO 5.7.2, for example, you should use `cisco-nso-nc-5.7`. - -These packages are found in the `$NCS_DIR/packages/lsa` directory. Each package contains the complete model of the `ncs` namespace for the corresponding NSO version, compiled as an LSA NED. Please always use the `cisco-nso` package included with the NSO version of the upper CFS node and not some older variant (such as the one from the lower RFS node) as it may not work correctly. - -Second, installing the cisco-nso LSA NED package will make the corresponding `ned-id` available, such as `cisco-nso-nc-5.7` (`ned-id` matches the package name). Use this `ned-id` for the RFS nodes instead of `lsa-netconf`. For example: - -```cli -admin@upper-nso% show devices device | display-level 4 -device lower-nso-1 { - lsa-remote-node lower-nso-1; - authgroup default; - device-type { - netconf { - ned-id cisco-nso-nc-5.7; - } - } - state { - admin-state unlocked; - } -} -``` - -This configuration allows the CFS node to communicate with a different NSO version but there are still some limitations. The upper CFS node must have the same or newer version than the managed RFS nodes. For all the currently supported versions of the lower node, the packages can be found in the `$NCS_DIR/packages/lsa` directory, but you may also be able to build an older one yourself. - -In case you already have a single-version deployment using the `lsa-netconf` `ned-id'`s, you can use the NED migrate procedure to switch to the new `ned-id` and multi-version deployment. - -### Device Compiled RFS Services - -Besides adding managed lower-layer nodes, the upper-layer node also requires packages for the services. Obviously, you must add the CFS package, which is an ordinary service package, to the CFS node. But you must also provide the device compiled RFS YANG models to allow provisioning of RFSs on the remote RFS nodes. - -The process resembles the way you create and compile device YANG models in normal NED packages. The `ncs-make-package` tool provides the `--lsa-netconf-ned` option, where you specify the location of the RFS YANG model and the tool creates a NED package for you. This is a new package that is separate from the RFS package used in the RFS nodes, so you might want to name it differently to avoid confusion. The following text uses the `-ned` suffix. - -Usually, you would also provide the `--no-netsim`, `--no-java`, and `--no-python` switches to the invocation, as the package is used with the NETCONF protocol and doesn't need any additional code. The `--no-netsim` option is required because netsim is not supported for these types of packages. For example: - -```bash -ncs-make-package --no-netsim --no-java --no-python \ - --lsa-netconf-ned ./path/to/rfs/src/yang \ - myrfs-service-ned -``` - -In this case, there is no explicit `--lsa-lower-nso` option specified and `ncs-make-package` will by default be set up to compile the package for the single version deployment, tied to the `lsa-netconf` `ned-id`. That means the models in the NED can be used with devices that have a `lsa-netconf` `ned-id` configured. - -To compile it for the multi-version deployment, which uses a different `ned-id`, you must select the target NSO version with the `--lsa-lower-nso cisco-nso-nc-X.Y` option, for example: - -```bash -ncs-make-package --no-netsim --no-java --no-python \ - --lsa-netconf-ned ./path/to/rfs/src/yang \ - --lsa-lower-nso cisco-nso-nc-5.7 - myrfs-service-ned -``` - -Depending on the RFS model, the package may fail to compile, even though the model compiles fine as a service. A typical error would indicate some node from a module, such as `tailf-ncs`, is not found. The reason is that the original RFS service YANG model has dependencies on other YANG models that are not included in the compilation process. - -One solution to this problem is to remove the dependencies in the YANG model before compilation. Normally this can be solved by changing the datatype in the NED compiled copy of the YANG model, for example from `leafref` or `instance-identifier` to string. This is only needed for the NED compiled copy, the lower RFS node YANG model can remain the same. There will then be an implicit conversion between types, at runtime, in the communication between the upper CFS node and the lower RFS node. - -An alternate solution, if you are doing a single version deployment and there are dependencies on the `tailf-ncs` namespace, is to switch to a multi-version deployment because the `cisco-nso` package includes this namespace (device compiled). Here, the NSO versions match but you are still using the `cisco-nso-nc-X.Y` `ned-id` and have to follow the instructions for the multi-version deployment. - -Once you have both, the CFS and device-compiled RFS service packages are ready; add them to the CFS node, then invoke a `sync-from` action to complete the setup process. - -### Example Walkthrough - -You can see all the required setup steps for a single version deployment performed in the example [examples.ncs/layered-services-architecture/lsa-single-version-deployment](https://github.com/NSO-developer/nso-examples/tree/6.6/layered-services-architecture/lsa-single-version-deployment) and the [examples.ncs/layered-services-architecture/lsa-multi-version-deployment](https://github.com/NSO-developer/nso-examples/tree/6.6/layered-services-architecture/lsa-multi-version-deployment) has the steps for the multi-version one. The two are quite similar but the multi-version deployment has additional steps, so it is the one described here. - -First, build the example for manual setup. - -```bash -$ make clean manual -$ make start-manual -$ make cli-upper-nso -``` - -Then configure the nodes in the cluster. This is needed so that the upper CFS node can receive notifications from the lower RFS node and prepare the upper CFS node to be used with the commit queue. - -```cli -> configure - -% set cluster device-notifications enabled -% set cluster remote-node lower-nso-1 authgroup default username admin -% set cluster remote-node lower-nso-1 address 127.0.0.1 port 2023 -% set cluster remote-node lower-nso-2 authgroup default username admin -% set cluster remote-node lower-nso-2 address 127.0.0.1 port 2024 -% set cluster commit-queue enabled -% commit -% request cluster remote-node lower-nso-* ssh fetch-host-keys -``` - -To be able to handle the lower NSO node as an LSA node, the correct version of the `cisco-nso-nc` package needs to be installed. In this example, 5.4 is used. - -Create a link to the `cisco-nso` package in the packages directory of the upper CFS node: - -```bash -$ ln -sf ${NCS_DIR}/packages/lsa/cisco-nso-nc-5.4 upper-nso/packages -``` - -Reload the packages: - -```cli -% exit -> request packages reload - -e>>> System upgrade is starting. ->>> Sessions in configure mode must exit to operational mode. ->>> No configuration changes can be performed until upgrade has completed. ->>> System upgrade has completed successfully. -reload-result { - package cisco-nso-nc-5.4 - result true -} -``` - -Now when the `cisco-nso-nc` package is in place, configure the two lower NSO nodes and `sync-from` them: - -```cli -> configure -Entering configuration mode private - -% set devices device lower-nso-1 device-type netconf ned-id cisco-nso-nc-5.4 -% set devices device lower-nso-1 authgroup default -% set devices device lower-nso-1 lsa-remote-node lower-nso-1 -% set devices device lower-nso-1 state admin-state unlocked -% set devices device lower-nso-2 device-type netconf ned-id cisco-nso-nc-5.4 -% set devices device lower-nso-2 authgroup default -% set devices device lower-nso-2 lsa-remote-node lower-nso-2 -% set devices device lower-nso-2 state admin-state unlocked - -% commit -Commit complete. - -% request devices fetch-ssh-host-keys -fetch-result { - device lower-nso-1 - result updated - fingerprint { - algorithm ssh-ed25519 - value 4a:c6:5d:91:6d:4a:69:7a:4e:0d:dc:4e:51:51:ee:e2 - } -} -fetch-result { - device lower-nso-2 - result updated - fingerprint { - algorithm ssh-ed25519 - value 4a:c6:5d:91:6d:4a:69:7a:4e:0d:dc:4e:51:51:ee:e2 - } -} - -% request devices sync-from -sync-result { - device lower-nso-1 - result true -} -sync-result { - device lower-nso-2 - result true -} -``` - -Now, for example, the configured devices of the lower nodes can be viewed: - -```cli -% show devices device config devices device | display xpath | display-level 5 - -/devices/device[name='lower-nso-1']/config/ncs:devices/device[name='ex0'] -/devices/device[name='lower-nso-1']/config/ncs:devices/device[name='ex1'] -/devices/device[name='lower-nso-1']/config/ncs:devices/device[name='ex2'] -/devices/device[name='lower-nso-2']/config/ncs:devices/device[name='ex3'] -/devices/device[name='lower-nso-2']/config/ncs:devices/device[name='ex4'] -/devices/device[name='lower-nso-2']/config/ncs:devices/device[name='ex5'] -``` - -Or, alarms inspected: - -```cli -% run show devices device lower-nso-1 live-status alarms summary - -live-status alarms summary indeterminates 0 -live-status alarms summary criticals 0 -live-status alarms summary majors 0 -live-status alarms summary minors 0 -live-status alarms summary warnings 0 -``` - -Now, create a netconf package on the upper CFS node which can be used towards the `rfs-vlan` service on the lower RFS node, in the shell terminal window, do the following: - -```bash -$ ncs-make-package --no-netsim --no-java --no-python \ - --lsa-netconf-ned package-store/rfs-vlan/src/yang \ - --lsa-lower-nso cisco-nso-nc-5.4 \ - --package-version 5.4 --dest upper-nso/packages/rfs-vlan-nc-5.4 \ - --build rfs-vlan-nc-5.4 -``` - -The created NED is an `lsa-netconf-ned` based on the YANG files of the `rfs-vlan` service: - -``` ---lsa-netconf-ned package-store/rfs-vlan/src/yang -``` - -The version of the NED reflects the version of the nso on the lower node: - -``` ---package-version 5.4 -``` - -The package will be generated in the packages directory of the upper NSO CFS node: - -``` ---dest upper-nso/packages/rfs-vlan-nc-5.4 -``` - -And, the name of the package will be: - -``` -rfs-vlan-nc-5.4 -``` - -Install the `cfs-vlan` service on the upper CFS node. In the shell terminal window, do the following: - -```bash -$ ln -sf ../../package-store/cfs-vlan upper-nso/packages -``` - -Reload the packages once more to get the `cfs-vlan` package. In the CLI terminal window, do the following: - -```cli -% exit - -> request packages reload - ->>> System upgrade is starting. ->>> Sessions in configure mode must exit to operational mode. ->>> No configuration changes can be performed until upgrade has completed. ->>> System upgrade has completed successfully. -reload-result { - package cfs-vlan - result true -} -reload-result { - package cisco-nso-nc-5.4 - result true -} -reload-result { - package rfs-vlan-nc-5.4 - result true -} - -> configure -Entering configuration mode private -``` - -Now, when all packages are in place a `cfs-vlan` service can be configured. The `cfs-vlan` service will dispatch service data to the right lower RFS node depending on the device names used in the service. - -In the CLI terminal window, verify the service: - -```cli -% set cfs-vlan v1 a-router ex0 z-router ex5 iface eth3 unit 3 vid 77 - -% commit dry-run -..... - local-node { - data devices { - device lower-nso-1 { - config { - services { - + vlan v1 { - + router ex0; - + iface eth3; - + unit 3; - + vid 77; - + description "Interface owned by CFS: v1"; - + } - } - } - } - device lower-nso-2 { - config { - services { - + vlan v1 { - + router ex5; - + iface eth3; - + unit 3; - + vid 77; - + description "Interface owned by CFS: v1"; - + } - } - } - } - } -..... -``` - -As `ex0` resides on `lower-nso-1` that part of the configuration goes there and the `ex5` part goes to `lower-nso-2`. - -### Migration and Upgrades - -Since an LSA deployment consists of multiple NSO nodes (or HA pairs of nodes), each can be upgraded to a newer NSO version separately. While that offers a lot of flexibility, it also makes upgrades more complex in many cases. For example, performing a major version upgrade on the upper CFS node only will make the deployment Multi-Version even if it was Single-Version before the upgrade, requiring additional action on your part. - -In general, staying with the Single-Version Deployment is the simplest option and does not require any further LSA-specific upgrade action (except perhaps recompiling the packages). However, the main downside is that, at least for a major upgrade, you must upgrade all the nodes at the same time (otherwise, you no longer have a Single-Version Deployment). - -If that is not feasible, the solution is to run a Multi-Version Deployment. Along with all of the requirements, the section [Multi-Version Deployment](layered-service-architecture.md#ncs_lsa.lsa_setup.multi_version) describes a major difference from the Single Version variant: the upper CFS node uses a version-specific `cisco-nso-nc-X.Y` NED ID to refer to lower RFS nodes. That means, if you switch to a Multi-Version Deployment, or perform a major upgrade of the lower-layer RFS node, the `ned-id` should change accordingly. However, do not change it directly but follow the correct NED upgrade procedure described in the section called [NED Migration](../management/ned-administration.md#sec.ned_migration). Briefly, the procedure consists of these steps: - -1. Keep the currently configured ned-id for an RFS device and the corresponding packages. If upgrading the CFS node, you will need to recompile the packages for the new NSO version. -2. Compile and load the packages that are device-compiled with the new `ned-id`, alongside the old packages. -3. Use the `migrate` action on a device to switch over to the new `ned-id`. - -The procedure requires you to have two versions of the device-compiled RFS service packages loaded in the upper CFS node when calling the `migrate` action: one version compiled by referencing the old (current) NED ID and the other one by referencing the new (target) NED ID. - -To illustrate, suppose you currently have an upper-layer and a lower-layer node both running NSO 5.4. The nodes were set up as described in the Single-Version Deployment option, with the upper CFS node using the `tailf-ncs-ned:lsa-netconf` NED ID for the lower-layer RFS node. The CFS node also uses the `rfs-vlan-ned` NED package for the `rfs-vlan` service. - -Now you wish to upgrade the CFS node to NSO 5.7 but keep the RFS node on the existing version 5.4. Before upgrading the CFS node, you create a backup and recompile the `rfs-vlan-ned` package for NSO 5.7. Note that the package references the `lsa-netconf` `ned-id`, which is the `ned-id` configured for the RFS device in the CFS node's CDB. Then, you perform the CFS node upgrade as usual. - -At this point the CFS node is running the new, 5.7 version and the RFS node is running 5.4. Since you now have a Multi-Version Deployment, you should migrate to the correct `ned-id` as well. Therefore, you prepare the `rfs-vlan-nc-5.4` package, as described in the Multi-Version Deployment option, compile the package, and load it into the CFS node. Thanks to the NSO CDM feature, both packages, `rfs-vlan-nc-5.4` and `rfs-vlan-ned`, can be used at the same time. - -With the packages ready, you execute the `devices device lower-nso-1 migrate new-ned-id cisco-nso-nc-5.4` command on the CFS node. The command configures the RFS device entry on CFS to use the new `cisco-nso-nc-5.4 ned-id`, as well as migrates the device configuration and service meta-data to the new model. Having completed the upgrade, you can now remove the `rfs-vlan-ned` if you wish. - -Later on, you may decide to upgrade the RFS node to NSO 5.6. Again, you prepare the new `rfs-vlan-nc-5.6` package for the CFS node in a similar way as before, now using the `cisco-nso-nc-5.6` ned-id instead of `cisco-nso-nc-5.4`. Next, you perform the RFS node upgrade to 5.6 and finally migrate the RFS device on the CFS node to the `cisco-nso-nc-5.6 ned-id`, with the `migrate` action. - -Likewise, you can return to the Single-Version Deployment, by upgrading the RFS node to the NSO 5.7, reusing the old, or preparing anew, the `rfs-vlan-ned` package and migrating to the `lsa-netconf ned-id`. - -All these `ned-id` changes stem from the fact that the upper-layer CFS node treats the lower-layer RFS node as a managed device, requiring the correct model, just like it does for any other device type. For the same reason, maintenance (bug fix or patch) NSO upgrades do not result in a changed `ned-id`, so for those, no migration is necessary. - -The [NSO example set](https://github.com/NSO-developer/nso-examples/tree/6.6/layered-services-architecture) illustrates different aspects of LSA deployment including working with single- and multi-version deployments. - -### User Authorization Passthrough - -In LSA, northbound users are authenticated on the CFS, and the request is re-authenticated on the RFS using either a system user or user/pass passthrough. - -For token-based authentication using external auth/package auth, this becomes a problem as the user and password are not expected to be locally provisioned and hence cannot be used for authentication towards the RFS, which leaves the option of a system user. - -Using a system user has two major limitations: - -* Auditing on the RFS becomes hard, as system sessions are not logged in the `audit.log`. -* Device-level RBAC becomes challenging as the devices reside in the RFS and the user information is lost. - -To handle this scenario, one can enable the passthrough of the user name and its groups to lower layer nodes to allow the session on the RFS to assume the same user as used on the CFS (similar to use of "sudo"). This will allow for the use of a system user between the CFS and RFS while allowing for auditing and RBAC on the RFS using the locally authenticated user on the CFS. - -On the CFS node, create an authgroup under `/devices/authgroups/group` with the `/devices/authgroups/group/{umap,default-map}/passthrough` empty leaf set, then select this authgroup on the configured RFS nodes by setting the `/devices/device/authgroup` leaf. When the passthrough leaf is set and a user (e.g., alice) on the CFS node connects to an RFS node, she will authenticate using the credentials specified in the `/devices/device/authgroup` authgroup (e.g., `lsa_passthrough_user` : `ahVaesai8Ahn0AiW`). Once the authentication completes successfully, the user `lsa_passthrough_user` changes into alice on the RFS node. - -{% code overflow="wrap" %} -```bash -admin@cfs% set devices authgroups group rfs-east default-map remote-name lsa_passthrough_user remote-password ahVaesai8Ahn0AiW passthrough -admin@cfs% set devices device rfs1 authgroup rfs-east -admin@cfs% set devices device rfs2 authgroup rfs-east -admin@cfs% commit -``` -{% endcode %} - -On the RFS node, configure the mapping of permitted users in the `/cluster/global-settings/passthrough/permit` list. The key of the permit list specifies what user may change into a different user. The different possible users to change into are specified by the `as-user` leaf-list, and the `as-group` leaf-list specifies valid groups. The user will end up with the intersection of groups in the user session on the CFS and the groups specified by the `as-group` leaf-list. Only users in the permit list will be allowed to change into the users set in the permit list elements `as-user` list. - -{% code overflow="wrap" %} -```bash -admin@rfs1% set cluster global-settings passthrough permit lsa_passthrough_user as-user [ alice bob carol ] as-group [ oper dev ] -admin@rfs1% commit -``` -{% endcode %} - -To allow the passthrough user to change into any user, set the `as-any-user` leaf, or for any group, set the `as-any-group` leaf. Use this with care as setting these leafs will allow the `lsa_passthrough_user` to elevate privileges by changing to `user admin` / `group admin`. - -{% code overflow="wrap" %} -```bash -admin@rfs1% set cluster global-settings passthrough permit lsa_passthrough_user as-any-user as-any-group -admin@rfs1% commit -``` -{% endcode %} diff --git a/administration/advanced-topics/locks.md b/administration/advanced-topics/locks.md deleted file mode 100644 index 41d86cfc..00000000 --- a/administration/advanced-topics/locks.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: Learn about different transaction locks in NSO and their interactions. ---- - -# Locks - -This section explains the different locks that exist in NSO and how they interact. It is important to understand the architecture of NSO with its management backplane and the transaction state machine as described in [Package Development](../../development/advanced-development/developing-packages.md) to be able to understand how the different locks fit into the picture. - -## Global Locks - -The NSO management backplane keeps a lock on the datastore running. This lock is usually referred to as the global lock, and it provides a mechanism to grant exclusive access to the datastore. - -The global is the only lock that can explicitly be taken through a northbound agent, for example, by the NETCONF `` operation, or by calling `Maapi.lock()`. - -A global lock can be taken for the whole datastore, or it can be a partial lock (for a subset of the data model). Partial locks are exposed through NETCONF and MAAPI and are only supported for operations toward the running datastore. - -An agent can request a global lock to ensure that it has exclusive write access. When a global lock is held by an agent, it is not possible for anyone else to write to the datastore that the lock guards—this is enforced by the transaction engine. A global lock on running is granted to an agent if there are no other holders of it (including partial locks) and if all data providers approve the lock request. Each data provider (CDB and/or external data providers) will have its `lock()` callback invoked to get a chance to refuse or accept the lock. The output of `ncs --status` includes the locking status. For each user session, locks (if any) per datastore, is listed. - -## Transaction Locks - -A northbound agent starts a user session towards NSO's management backplane. Each user session can then start multiple transactions. A transaction is either read/write or read-only. - -The transaction engine has its internal locks towards the running datastore. These transaction locks exist to serialize configuration updates towards the datastore and are separate from the global locks. - -As a northbound agent wants to update the running datastore with a new configuration, it will implicitly grab and release the transactional lock. The transaction engine takes care of managing the locks as it moves through the transaction state machine, and there is no API that exposes the transactional locks to the northbound agents. - -When the transaction engine wants to take a lock for a transaction (for example, when entering the validate state), it first checks that no other transaction has the lock. Then it checks that no user session has a global lock on that datastore. Finally, each data provider is invoked by its `transLock()` callback. - -## Northbound Agents and Global Locks - -In contrast to the implicit transactional locks, some northbound agents expose explicit access to the global locks. This is done a bit differently by each agent. - -The management API exposes the global locks by providing `Maapi.lock()` and `Maapi.unlock()` methods (and the corresponding `Maapi.lockPartial()` `Maapi.unlockPartial()` for partial locking). Once a user session is established (or attached to), these functions can be called. - -In the CLI, the global locks are taken when entering different configure modes as follows: - -* `config exclusive`: The running datastore global lock will be taken. -* `config terminal`: Does not grab any locks. - -The global lock is then kept by the CLI until the configure mode is exited. - -The Web UI behaves in the same way as the CLI (it presents three edit tabs called **Edit private**, **Edit exclusive**, and which correspond to the CLI modes described above). - -The NETCONF agent translates the `` operation into a request for the global lock for the requested datastore. Partial locks are also exposed through the partial-lock RPC. - -## External Data Providers - -Implementing the `lock()` and `unlock()` callbacks is not required of an external data provider. NSO will never try to initiate the `transLock()` state transition (see the transaction state diagram in [Package Development](../../development/advanced-development/developing-packages.md)) towards a data provider while a global lock is taken—so the reason for a data provider to implement the locking callbacks is if someone else can write (or lock, for example, to take a backup) to the data provider's database. - -## CDB and Locks - -CDB ignores the `lock()` and `unlock()` callbacks (since the data-provider interface is the only write interface towards it). - -CDB has its own internal locks on the database. The running datastore has a single write and multiple read locks. It is not possible to grab the write lock on a datastore while there are active read locks on it. The locks in CDB exist to make sure that a reader always gets a consistent view of the data (in particular, it becomes very confusing if another user is able to delete configuration nodes in between calls to `getNext()` on YANG list entries). - -During a transaction, `transLock()` takes a CDB read lock towards the transaction's datastore, and `writeStart()` tries to release the read lock and grab the write lock instead. - -A CDB external reader client implicitly takes a CDB read lock between `Cdb.startSession()` and `Cdb.endSession()` This means that while a CDB client is reading, a transaction can not pass through `writeStart()` (and conversely, a CDB reader can not start while a transaction is in between `writeStart()` and `commit()` or `abort()`). - -The operational store in CDB does not have any locks. NSO's transaction engine can only read from it, and the CDB client writes are atomic per write operation. - -## Lock Impact on User Sessions - -When a session tries to modify a data store that is locked in some way, it will fail. For example, the CLI might print: - -```bash -admin@ncs(config)# commit -Aborted: the configuration database is locked -``` - -Since some of the locks are short-lived (such as a CDB read-lock), NSO is by default configured to retry the failing operation for a short period of time. If the data store still is locked after this time, the operation fails. - -To configure this, set `/ncs-config/commit-retry-timeout` in `ncs.conf`. diff --git a/administration/advanced-topics/restart-strategies-for-service-manager.md b/administration/advanced-topics/restart-strategies-for-service-manager.md deleted file mode 100644 index d80a2ac8..00000000 --- a/administration/advanced-topics/restart-strategies-for-service-manager.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -description: Restart strategy for the service manager. ---- - -# Service Manager Restart - -The service manager executes in a Java VM outside of NSO. The `NcsMux` initializes a number of sockets to NSO at startup. These are Maapi sockets and data provider sockets. NSO can choose to close any of these sockets whenever NSO requests the service manager to perform a task, and that task is not finished within the stipulated timeout. If that happens, the service manager must be restarted. The timeout(s) are controlled by several `ncs.conf` parameters found under `/ncs-config/japi`. diff --git a/administration/get-started.md b/administration/get-started.md deleted file mode 100644 index 0535e407..00000000 --- a/administration/get-started.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -description: Administrate and manage NSO. -icon: chevrons-right ---- - -# Get Started - -## Installation and Deployment - -
Local InstallInstall NSO for test and evaluation use.local-install.md
System InstallInstall NSO for production system-wide use.system-install.md
Post-install ActionsPerform post-install actions after installing NSO.post-install-actions
Containerized NSODeploy NSO using Cisco-provided container images.containerized-nso.md
Dev to Prod DeploymentDeploy NSO from development to production.development-to-production-deployment
Upgrade NSOUpgrade NSO installation to a higher version.upgrade-nso.md
- -## Management - -
System ManagementConfigure & manage your NSO deployment.system-management
Package ManagementLearn about NSO packages and how to use them.package-mgmt.md
High AvailabilitySet up multiple nodes in a highly-available (HA) setup.high-availability.md
AAA InfrastructureSet up user authentication and authorization.aaa-infrastructure.md
NED AdministrationAdminister and manage Cisco-provided NEDs.ned-administration.md
- -## Advanced Topics - -
LocksUnderstand how transaction locks work.locks.md
CDB PersistenceSelect the optimal CDB persistence mode.cdb-persistence.md
IPC ConnectionLearn how client libraries connect to NSO.ipc-connection.md
Cryptographic KeysEncrypt and decrypt strings in NSO using crypto keys.cryptographic-keys.md
Service Manager RestartConfigure the timeout period of Service Manager.restart-strategies-for-service-manager.md
IPv6 on NorthboundUse IPv6 on Northbound NSO interfaces.ipv6-on-northbound-interfaces.md
LSALearn about Layered Service Architecture.layered-service-architecture.md
diff --git a/administration/installation-and-deployment/README.md b/administration/installation-and-deployment/README.md deleted file mode 100644 index e3c2356d..00000000 --- a/administration/installation-and-deployment/README.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -description: Learn about different ways to install and deploy NSO. -icon: download ---- - -# Installation and Deployment - -## Ways to Deploy NSO - -* [By installation](./#by-installation) -* [By using Cisco-provided container images](./#by-using-cisco-provided-container-images) - -### By Installation - -Choose this way if you want to install NSO on a system. Before proceeding with the installation, decide on the install type. - -#### Install Types - -The installation of NSO comes in two variants. - -{% hint style="info" %} -Both variants can be installed in **standard mode** or in [**FIPS**](https://www.nist.gov/itl/publications-0/federal-information-processing-standards-fips)**-compliant** mode. See the detailed installation instructions for more information. -{% endhint %} - -
Local InstallLocal Install is used for development, lab, and evaluation purposes. It unpacks all the application components, including docs and examples. It can be used by the engineer to run multiple, unrelated, instances of NSO for different labs and demos on a single workstation.local-install.md
System InstallSystem Install is used when installing NSO for a centralized, always-on, production-grade, system-wide deployment. It is configured as a system daemon that would start and end with the underlying operating system. The default users of admin and operator are not included and the file structure is more distributed.system-install.md
- -{% hint style="info" %} -All the NSO examples and README steps provided with the installation are based on and intended for Local Install only. Use Local Install for evaluation and development purposes only. - -System Install should be used only for production deployment. For all other purposes, use the Local Install procedure. -{% endhint %} - -### By Using Cisco-Provided Container Images - -Choose this way if you want to run NSO in a container, such as Docker. Visit the link below for more information. - -{% content-ref url="containerized-nso.md" %} -[containerized-nso.md](containerized-nso.md) -{% endcontent-ref %} - -*** - -> **Supporting Information** -> -> If you are evaluating NSO, you should have a designated support contact. If you have an NSO support agreement, please use the support channels specified in the agreement. In either case, do not hesitate to reach out to us if you have questions or feedback. diff --git a/administration/installation-and-deployment/containerized-nso.md b/administration/installation-and-deployment/containerized-nso.md deleted file mode 100644 index da4045f6..00000000 --- a/administration/installation-and-deployment/containerized-nso.md +++ /dev/null @@ -1,870 +0,0 @@ ---- -description: Deploy NSO in a containerized setup using Cisco-provided images. ---- - -# Containerized NSO - -NSO can be deployed in your environment using a container, such as Docker. Cisco offers two pre-built images for this purpose that you can use to run NSO and build packages (see [Overview of NSO Images](containerized-nso.md#d5e8294)). - -*** - -**Migration Information** - -If you are migrating from an existing NSO System Install to a container-based setup, follow the guidelines given below in [Migration to Containerized NSO](containerized-nso.md#sec.migrate-to-containerizednso). - -*** - -## Use Cases for Containerized Approach - -Running NSO in a container offers several benefits that you would generally expect from a containerized approach, such as ease of use and convenient distribution. More specifically, a containerized NSO approach allows you to: - -* Run a container image of a specific version of NSO and your packages which can then be distributed as one unit. -* Deploy and distribute the same version across your production environment. -* Use the Build Image containing the necessary environment for compiling NSO packages. - -## Overview of NSO Images - -Cisco provides the following two NSO images based on Red Hat UBI. - -* [Production Image](containerized-nso.md#production-image) -* [Build Image](containerized-nso.md#build-image) - -
Intended UseDevelop NSO PackagesBuild NSO PackagesRun NSONSO Install Type
Development HostNone or Local Install
Build ImageSystem Install
Production ImageSystem Install
- -{% hint style="info" %} -The Red Hat UBI is an OCI-compliant image that is freely distributable and independent of platform and technical dependencies. You can read more about Red Hat UBI [here](https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image), and about Open Container Initiative (OCI) [here](https://opencontainers.org/faq/). -{% endhint %} - -### Production Image - -The Production Image is a production-ready NSO image for system-wide deployment and use. It is based on NSO [System Install](system-install.md) and is available from the [Cisco Software Download](https://software.cisco.com/download/home) site. - -Use the pre-built image as the base image in the container file (e.g., Dockerfile) and mount your own packages (such as NEDs and service packages) to run a final image for your production environment (see examples below). - -{% hint style="info" %} -Consult the [Installation](./) documentation for information on installing NSO on a Docker host, building NSO packages, etc. -{% endhint %} - -{% hint style="info" %} -See [Developing and Deploying a Nano Service](deployment/develop-and-deploy-a-nano-service.md) for an example that uses the container to deploy an SSH-key-provisioning nano service. - -The README in the [examples.ncs/getting-started/netsim-sshkey](https://github.com/NSO-developer/nso-examples/tree/6.6/getting-started/netsim-sshkey) example provides a link to the container-based deployment variant of the example. See the `setup_ncip.sh` script and `README` in the `netsim-sshkey` deployment example for details. -{% endhint %} - -### Build Image - -The Build Image is a separate standalone NSO image with the necessary environment and software for building packages. It is provided specifically to address the developer needs of building packages. - -The image is available as a signed package (e.g., `nso-VERSION.container-image-build.linux.ARCH.signed.bin`) from the Cisco [Software Download](https://software.cisco.com/download/home) site. You can run the Build Image in different ways, and a simple tool for defining and running multi-container Docker applications is [Docker Compose](https://docs.docker.com/compose/) (see examples below). - -The container provides the necessary environment to build custom packages. The Build Image adds a few Linux packages that are useful for development, such as Ant, JDK, net-tools, pip, etc. Additional Linux packages can be added using, for example, the `dnf` command. The `dnf list installed` command lists all the installed packages. - -## Downloading and Extracting the Images - -To fetch and extract NSO images: - -1. On Cisco's official [Software Download](https://software.cisco.com/download/home) site, search for "Network Services Orchestrator". Select the relevant NSO version in the drop-down list, e.g., "Crosswork Network Services Orchestrator 6"**,** and click "Network Services Orchestrator Software". Locate the binary, which is delivered as a signed package (e.g., `nso-6.4.container-image-prod.linux.x86_64.signed.bin`). -2. Extract the image and other files from the signed package, for example: - - ```bash - sh nso-6.4.container-image-prod.linux.x86_64.signed.bin - ``` - -{% hint style="info" %} -**Signed Archive File Pattern** - -The signed archive file name has the following pattern: - -`nso-VERSION.container-image-PROD_BUILD.linux.ARCH.signed.bin`, where: - -* `VERSION` denotes the image's NSO version. -* `PROD_BUILD` denotes the type of the container (i.e., `prod` for Production, and `build` for Build). -* `ARCH` is the CPU architecture. -{% endhint %} - -## System Requirements - -To run the images, make sure that your system meets the following requirements: - -* A system running Linux `x86_64` or `ARM64`, or macOS `x86_64` or Apple Silicon. Linux for production. -* A container platform. Docker is the recommended platform and is used as an example in this guide for running NSO images. You may use another container runtime of your choice. Note that commands in this guide are Docker-specific. if you use another container runtime, make sure to use the respective commands. -* To check the Java (JDK) and Python versions included in the container, use the following command, (where `cisco-nso-prod:6.5` is the image you want to check): - - {% code title="Example: Check Java and Python Versions of Container" %} - ```bash - docker run --rm cisco-nso-prod:6.5 sh -c "java -version && python --version" - ``` - {% endcode %} - -{% hint style="info" %} -Docker on Mac uses a Linux VM to run the Docker engine, which is compatible with the normal Docker images built for Linux. You do not need to recompile your NSO-in-Docker images when moving between a Linux machine and Docker on Mac as they both essentially run Docker on Linux. -{% endhint %} - -## Administrative Information - -This section covers the necessary administrative information about the NSO Production Image. - -### Migrate to Containerized NSO Setup - -If you have NSO installed as a System Install, you can migrate to the Containerized NSO setup by following the instructions in this section. Migrating your Network Services Orchestrator (NSO) to a containerized setup can provide numerous benefits, including improved scalability, easier version management, and enhanced isolation of services. - -The migration process is designed to ensure a smooth transition from a System-Installed NSO to a container-based deployment. Detailed steps guide you through preparing your existing environment, exporting the necessary configurations and state data, and importing them into your new containerized NSO instance. During the migration, consider the container runtime you plan to use, as this impacts the migration process. - -**Before You Start** - -* We recommend reading through this guide to understand better the expectations, requirements, and functioning aspects of a containerized deployment. -* Verify the compatibility of your current system configurations with the containerized NSO setup. See [System Requirements](containerized-nso.md#sec.system-reqs) for more information. -* Note that [NSO runs from a non-root user ](containerized-nso.md#nso-runs-from-a-non-root-user)with the containerized NSO setup[.](containerized-nso.md#nso-runs-from-a-non-root-user) -* Determine and install the container orchestration tool you plan to use (e.g., Docker, etc.). -* Ensure that your current NSO installation is fully operational and backed up and that you have a clear rollback strategy in case any issues arise. Pay special attention to customizations and integrations that your current NSO setup might have, and verify their compatibility with the containerized version of NSO. -* Have a contingency plan in place for quick recovery in case any issues are encountered during migration. - -**Migration Steps** - -Prepare: - -1. Document your current NSO environment's specifics, including custom configurations and packages. -2. Perform a complete backup of your existing NSO instance, including configurations, packages, and data. -3. Set up the container environment and download/extract the NSO production image. See [Downloading and Extracting the Images](containerized-nso.md#sec.fetch-images) for details. - -Migrate: - -1. Stop the current NSO instance. -2. Save the run directory from the NSO instance in an appropriate place. -3. Use the same `ncs.conf` and High Availability (HA) setup previously used with your System Install. We assume that the `ncs.conf` follows the best practice and uses the `NCS_DIR`, `NCS_RUN_DIR`, `NCS_CONFIG_DIR`, and `NCS_LOG_DIR` variables for all paths. The `ncs.conf` can be added to a volume and mounted to `/nso/etc` in the container. - - ```bash - docker container create --name temp -v NSO-evol:/nso/etc hello-world - docker cp ncs.conf temp:/nso/etc - docker rm temp - ``` -4. Add the run directory as a volume, mounted to `/nso/run` in the container and copy the CDB data, packages, etc., from the previous System Install instance. - - ```bash - cd path-to-previous-run-dir - docker container create --name temp -v NSO-rvol:/nso/run hello-world - docker cp . temp:/nso/run - docker rm temp - ``` -5. Create a volume for the log directory. - - ```bash - docker volume create --name NSO-lvol - ``` -6. Start the container. Example: - - ```bash - docker run -v NSO-rvol:/nso/run -v NSO-evol:/nso/etc -v NSO-lvol:/log -itd \ - --name cisco-nso -e EXTRA_ARGS=--with-package-reload -e ADMIN_USERNAME=admin \ - -e ADMIN_PASSWORD=admin cisco-nso-prod:6.4 - ``` - -Finalize: - -1. Ensure that the containerized NSO instance functions as expected and validate system operations. -2. Plan and execute your cutover transition from the System-Installed NSO to the containerized version with minimal disruption. -3. Monitor the new setup thoroughly to ensure stability and performance. - -### `ncs.conf` File Configuration and Preference - -The `run-nso.sh` script runs a check at startup to determine which `ncs.conf` file to use. The order of preference is as below: - -1. The `ncs.conf` file specified in the Dockerfile (i.e., `ENV $NCS_CONFIG_DIR /etc/ncs/`) is used as the first preference. -2. The second preference is to use the `ncs.conf` file mounted in the `/nso/etc/` run directory. -3. If no `ncs.conf` file is found at either `/etc/ncs` or `/nso/etc`, the default `ncs.conf` file provided with the NSO image in `/defaults` is used. - -{% hint style="info" %} -If the `ncs.conf` file is edited after startup, it can be reloaded using MAAPI `reload_config()`. Example: `$ ncs_cmd -c "reload"`. -{% endhint %} - -{% hint style="info" %} -The default `ncs.conf` file in `/defaults` has a set of environment variables that can be used to enable interfaces (all interfaces are disabled by default) which is useful when spinning up the Production container for quick testing. An interface can be enabled by setting the corresponding environment variable to `true`. - -* `NCS_CLI_SSH`: Enables CLI over SSH on port `2024`. -* `NCS_WEBUI_TRANSPORT_TCP`: Enables JSON-RPC and RESTCONF over TCP on port `8080`. -* `NCS_WEBUI_TRANSPORT_SSL`: Enables JSON-RPC and RESTCONF over SSL/TLS on port `8888`. -* `NCS_NETCONF_TRANSPORT_SSH`: Enables NETCONF over SSH on port `2022`. -* `NCS_NETCONF_TRANSPORT_TCP`: Enables NETCONF over TCP on port `2023`. -{% endhint %} - -### Pre- and Post-Start Scripts - -If you need to perform operations before or after the `ncs` process is started in the Production container, you can use Python and/or Bash scripts to achieve this. Add the scripts to the `$NCS_CONFIG_DIR/pre-ncs-start.d/` and `$NCS_CONFIG_DIR/post-ncs-start.d/` directories to have the `run-nso.sh` script run them. - -### NSO Runs from a Non-Root User - -NSO is installed with the `--run-as-user` option for build and production containers to run NSO from the non-root `nso` user that belongs to the `nso` user group. - -When migrating from container versions where NSO has `root` privilege, ensure the `nso` user owns or has access rights to the required files and directories. Examples include application directories, SSH host keys, SSH keys used to authenticate with devices, etc. See the deployment example variant referenced by the [examples.ncs/getting-started/netsim-sshkey/README.md](https://github.com/NSO-developer/nso-examples/tree/6.6/getting-started/netsim-sshkey) for an example. - -The NSO container runs a script called `take-ownership.sh` as part of its startup, which takes ownership of all the directories that NSO needs. The script will be one of the first things to run. The script can be overridden to take ownership of even more directories, such as mounted volumes or bind mounts. - -### Admin User Creation - -An admin user can be created on startup by the run script in the container. Three environment variables control the addition of an admin user: - -* `ADMIN_USERNAME`: Username of the admin user to add, default is `admin`. -* `ADMIN_PASSWORD`: Password of the admin user to add. -* `ADMIN_SSHKEY`: Private SSH key of the admin user to add. - -As `ADMIN_USERNAME` already has a default value, only `ADMIN_PASSWORD`, or `ADMIN_SSHKEY` need to be set in order to create an admin user. For example: - -```bash -docker run -itd --name cisco-nso -e ADMIN_PASSWORD=admin cisco-nso-prod:6.4 -``` - -This can be useful when starting up a container in CI for testing or development purposes. It is typically not required in a production environment where CDB already contains the required user accounts. - -{% hint style="info" %} -When using a permanent volume for CDB, and restarting the NSO container multiple times with a different `ADMIN_USERNAME` or `ADMIN_PASSWORD`, the start script uses these environment variables to generate an XML file named `add_admin_user.xml`. The generated XML file is added to the CDB directory to be read at startup. But if the persisted CDB configuration file already exists in the CDB directory, NSO will not load any XML files at startup, instead the generated `add_admin_user.xml` in the CDB directory needs to be loaded manually. -{% endhint %} - -{% hint style="info" %} -The default `ncs.conf` file performs authentication using only the Linux PAM, with local authentication disabled. For the `ADMIN_USERNAME`, `ADMIN_PASSWORD`, and `ADMIN_SSHKEY` variables to take effect, NSO's local authentication, in `/ncs-conf/aaa/local-authentication`, needs to be enabled. Alternatively, you can create a local Linux admin user that is authenticated by NSO using Linux PAM. -{% endhint %} - -### Exposing Ports - -The default `ncs.conf` NSO configuration file does not enable any northbound interfaces, and no ports are exposed externally to the container. Ports can be exposed externally of the container when starting the container with the northbound interfaces and their ports correspondingly enabled in `ncs.conf`. - -### Backup and Restore - -The backup behavior of running NSO in vs. outside the container is largely the same, except that when running NSO in a container, the SSH and SSL certificates are not included in the backup produced by the `ncs-backup` script. This is different from running NSO outside a container where the default configuration path `/etc/ncs` is used to store the SSH and SSL certificates, i.e., `/etc/ncs/ssh` for SSH and `/etc/ncs/ssl` for SSL. - -**Take a Backup** - -Let's assume we start a production image container using: - -```bash -docker run -d --name cisco-nso -v NSO-vol:/nso -v NSO-log-vol:/log cisco-nso-prod:6.4 -``` - -To take a backup: - -* Run the `ncs-backup` command. The backup file is written to `/nso/run/backups`. - - ```bash - docker exec -it cisco-nso ncs-backup - INFO Backup /nso/run/backups/ncs-6.4@2024-11-03T11:31:07.backup.gz created successfully - ``` - -**Restore a Backup** - -To restore a backup, NSO must be stopped. As you likely only have access to the `ncs-backup` tool, the volume containing CDB and other run-time data from inside of the NSO container, this poses a slight challenge. Additionally, shutting down NSO will terminate the NSO container. - -To restore a backup: - -1. Shut down the NSO container: - - ```bash - docker stop cisco-nso - docker rm cisco-nso - ``` -2. Run the `ncs-backup --restore` command. Start a new container with the same persistent shared volumes mounted but with a different command. Instead of running the `/run-nso.sh`, which is the normal command of the NSO container, run the `restore` command. - - ```bash - docker run -u root -it --rm -v NSO-vol:/nso -v NSO-log-vol:/log \ - --entrypoint ncs-backup cisco-nso-prod:6.4 \ - --restore /nso/run/backups/ncs-6.4@2024-11-03T11:31:07.backup.gz - - Restore /etc/ncs from the backup (y/n)? y - Restore /nso/run from the backup (y/n)? y - INFO Restore completed successfully - ``` -3. Restoring an NSO backup should move the current run directory (`/nso/run` to `/nso/run.old`) and restore the run directory from the backup to the main run directory (`/nso/run`). After this is done, start the regular NSO container again as usual.\\ - - ```bash - docker run -d --name cisco-nso -v NSO-vol:/nso -v NSO-log-vol:/log cisco-nso-prod:6.4 - ``` - -### SSH Host Key - -The NSO image `/run-nso.sh` script looks for an SSH host key named `ssh_host_ed25519_key` in the `/nso/etc/ssh` directory to be used by the NSO built-in SSH server for the CLI and NETCONF interfaces. - -If an SSH host key exists, which is for a typical production setup stored in a persistent shared volume, it remains the same after restarts or upgrades of NSO. If no SSH host key exists, the script generates a private and public key. - -In a high-availability (HA) setup, the host key is typically shared by all NSO nodes in the HA group and stored in a persistent shared volume. This is done to avoid fetching the public host key from the new primary after each failover. - -### HTTPS TLS Certificate - -NSO expects to find a TLS certificate and key at `/nso/ssl/cert/host.cert` and `/nso/ssl/cert/host.key` respectively. Since the `/nso` path is usually on persistent shared volume for production setups, the certificate remains the same across restarts or upgrades. - -If no certificate is present, one will be generated. It is a self-signed certificate valid for 30 days making it possible to use both in development and staging environments. It is not meant for the production environment. You should replace it with a properly signed certificate for production and it is encouraged to do so even for test and staging environments. Simply generate one and place it at the provided path, for example using the following, which is the command used to generate the temporary self-signed certificate: - -``` -openssl req -new -newkey rsa:4096 -x509 -sha256 -days 30 -nodes \ --out /nso/ssl/cert/host.cert -keyout /nso/ssl/cert/host.key \ --subj "/C=SE/ST=NA/L=/O=NSO/OU=WebUI/CN=Mr. Self-Signed" -``` - -### YANG Model Changes (destructive) - -The database in NSO, called CDB, uses YANG models as the schema for the database. It is only possible to store data in CDB according to the YANG models that define the schema. - -If the YANG models are changed, particularly if the nodes are removed or renamed (rename is the removal of one leaf and an addition of another), any data in CDB for those leaves will also be removed. NSO normally warns about this when you attempt to load new packages, for example, `request packages reload` command refuses to reload the packages if the nodes in the YANG model have disappeared. You would then have to add the **force** argument, e.g., `request packages reload force`. - -### Health Check - -The base Production Image comes with a basic container health check. It uses `ncs_cmd` to get the state that NCS is currently in. Only the result status is observed to check if `ncs_cmd` was able to communicate with the `ncs` process. The result indicates if the `ncs` process is responding to IPC requests. - -{% hint style="info" %} -The default `--health-start-period duration` in health check is set to 60 seconds. NSO will report an `unhealthy` state if it takes more than 60 seconds to start up. To resolve this, set the `--health-start-period duration` value to a relatively higher value, such as 600 seconds, or however long you expect NSO will take to start up. - -To disable the health check, use the `--no-healthcheck` command. -{% endhint %} - -### NSO System Dump and Enable Strict Overcommit Accounting on the Host - -By default, the Linux kernel allows overcommit of memory. However, memory overcommit produces an unexpected and unreliable environment for NSO since the Linux Out‑Of‑Memory (OOM) killer may terminate NSO without restarting it if the system is critically low on memory. - -Also, when the OOM-killer terminates NSO, NSO will not produce a system dump file, and the debug information will be lost. Thus, it is strongly recommended that overcommit is disabled with Linux NSO production container hosts with an overcommit ratio of less than 100% (max). Use a 5% headroom (overcommit\_ratio≈95 when no swap) or increase if the host runs additional services. Or use vm.overcommit\_kbytes for a fixed CommitLimit. - -See [Step - 4. Run the Installer](system-install.md#si.run.the.installer) in System Install for information on memory overcommit recommendations for a Linux system hosting NSO production containers. - -{% hint style="info" %} -By default, NSO writes a system dump to the NSO run-time directory, default `NCS_RUN_DIR=/nso/run`. If the `NCS_RUN_DIR` is not pointing to a persistent, host‑mounted volume so dumps survive container restarts or to give the NSO system dump file a unique name, the `NCS_DUMP="/path/to/mounted/dir/ncs_crash.dump.$(date +%Y%m%d-%H%M%S)"` variable needs to be set. -{% endhint %} - -#### Recommended: Host Configured for Strict Overcommit - -With the host configured for strict overcommit (`vm.overcommit_memory=2`), containers inherit the host’s CommitLimit behavior. Note that `vm.overcommit_memory`, `vm.overcommit_ratio`, and `vm.overcommit_kbytes` are host‑global and cannot be set per container. These `vm.*` settings are configured on the host and apply to all containers. - -* Optionally use the `docker run` command to set memory limits and swap: - * Use `--memory=` to cap the container’s RAM. - * Set `--memory-swap=` equal to `--memory` to effectively disable swap for the container. - * If swap must be enabled, use a fast disk, for example, an NVMe SSD. - -#### **Alternative: Heuristic Overcommit Mode** - -The alternative, using heuristic overcommit mode, can be useful if the NSO host has severe memory limitations. For example, if RAM sizing for the NSO host did not take into account that the schema (from YANG models) is loaded into memory by NSO Python and Java packages affecting total committed memory (Committed\_AS) and after considering the recommendations in [CDB Stores the YANG Model Schema](../../development/advanced-development/scaling-and-performance-optimization.md#d5e8743). - -As an alternative to the recommended strict mode, `vm.overcommit_memory=2`, you can keep `vm.overcommit_memory=0` configured on the host to allow overcommit of memory and trigger `ncs --debug-dump` when Committed\_AS reaches, for example, 95% of CommitLimit or when the container’s cgroup memory usage reaches, for example, 90% of its cap. - -* This approach does not prevent the Linux OOM-killer from killing NSO or the container; it only attempts to capture diagnostic data before memory pressure becomes critical. OOM kills can occur even when Committed\_AS < CommitLimit due to cgroup limits or reclaim failure. -* The same `docker run` memory and swap options as above can be used. -* Monitor the Committed\_AS vs CommitLimit and cgroup memory usage vs cap using, for example, a script or an observability tool. - * Note that Committed\_AS and CommitLimit from `/proc/meminfo` are host‑wide values. Inside a container, they reflect the host, not the container’s cgroup budget. - * cgroup memory.current vs memory.max is the primary predictor for container OOM events; the host CommitLimit is an additional early‑warning signal. -* Ensure the user running the monitor has permission to execute `ncs --debug-dump` and write to the chosen dump directory. - -{% code title="Simple example of an NSO debug-dump monitor inside a container" overflow="wrap" %} -```bash -#!/usr/bin/env bash -# Simple NSO debug-dump monitor inside a container (vm.overcommit_memory=0 on host). -# Triggers ncs --debug-dump when Committed_AS reaches 95% of CommitLimit -# or when the container’s cgroup memory usage reaches 90% of its cap. - -THRESHOLD_PCT=95 # CommitLimit threshold (5% headroom). -CGROUP_THRESHOLD_PCT=90 # Trigger when memory.current >= 90% of memory.max. -POLL_INTERVAL=5 # Seconds between checks. -PROCESS_CHECK_INTERVAL=30 -DUMP_COUNT=10 -DUMP_DELAY=10 -DUMP_PREFIX="dump" - -command -v ncs >/dev/null 2>&1 || { echo "ncs command not found in PATH."; exit 1; } - -find_nso_pid() { - pgrep -x ncs.smp | head -n1 || true -} - -read_cgroup_mem_kb() { - # Outputs: current_kb max_kb (max_kb=0 if unlimited or not found) - if [ -r /sys/fs/cgroup/memory.current ]; then - local cur max - cur=$(cat /sys/fs/cgroup/memory.current 2>/dev/null) - max=$(cat /sys/fs/cgroup/memory.max 2>/dev/null) - [ "$max" = "max" ] && max=0 - echo "$((cur/1024)) $((max/1024))" - else - echo "0 0" - fi -} - -while true; do - pid="$(find_nso_pid)" - if [ -z "${pid:-}" ]; then - echo "NSO not running; retry in ${PROCESS_CHECK_INTERVAL}s..." - sleep "$PROCESS_CHECK_INTERVAL" - continue - fi - - committed="$(awk '/Committed_AS:/ {print $2}' /proc/meminfo)" - commit_limit="$(awk '/CommitLimit:/ {print $2}' /proc/meminfo)" - if [ -z "$committed" ] || [ -z "$commit_limit" ]; then - echo "Unable to read /proc/meminfo; retry in ${POLL_INTERVAL}s..." - sleep "$POLL_INTERVAL" - continue - fi - - threshold=$(( commit_limit * THRESHOLD_PCT / 100 )) - read cg_current_kb cg_max_kb < <(read_cgroup_mem_kb) - cgroup_trigger=0 - if [ "${cg_max_kb:-0}" -gt 0 ]; then - cgroup_pct=$(( cg_current_kb * 100 / cg_max_kb )) - [ "$cgroup_pct" -ge "$CGROUP_THRESHOLD_PCT" ] && cgroup_trigger=1 - echo "PID=${pid} Committed_AS=${committed}kB; CommitLimit=${commit_limit}kB; Threshold=${threshold}kB; cgroup=${cg_current_kb}kB/${cg_max_kb}kB (${cgroup_pct}%)." - else - echo "PID=${pid} Committed_AS=${committed}kB; CommitLimit=${commit_limit}kB; Threshold=${threshold}kB; cgroup=unlimited." - fi - - if [ "$committed" -ge "$threshold" ] || [ "$cgroup_trigger" -eq 1 ]; then - echo "Threshold crossed; collecting ${DUMP_COUNT} debug dumps..." - for i in $(seq 1 "$DUMP_COUNT"); do - file="${DUMP_PREFIX}.${i}.bin" - echo "Dump $i -> ${file}" - if ! ncs --debug-dump "$file"; then - echo "Debug dump $i failed." - fi - sleep "$DUMP_DELAY" - done - echo "All debug dumps completed; exiting." - exit 0 - fi - - sleep "$POLL_INTERVAL" -done -``` -{% endcode %} - -### Startup Arguments - -The `/nso-run.sh` script that starts NSO is executed as an `ENTRYPOINT` instruction and the `CMD` instruction can be used to provide arguments to the entrypoint-script. Another alternative is to use the `EXTRA_ARGS` variable to provide arguments. The `/nso-run.sh` script will first check the `EXTRA_ARGS` variable before the `CMD` instruction. - -An example using `docker run` with the `CMD` instruction: - -```bash -docker run --name nso -itd cisco-nso-prod:6.4 --with-package-reload \ ---ignore-initial-validation -``` - -With the `EXTRA_ARGS` variable: - -```bash -docker run --name nso \ --e EXTRA_ARGS='--with-package-reload --ignore-initial-validation' \ --itd cisco-nso-prod:6.4 -``` - -An example using a Docker Compose file, `compose.yaml`, with the `CMD` instruction: - -``` -services: - nso: - image: cisco-nso-prod:6.4 - container_name: nso - command: - - --with-package-reload - - --ignore-initial-validation -``` - -With the `EXTRA_ARGS` variable: - -``` -services: - nso: - image: cisco-nso-prod:6.4 - container_name: nso - environment: - - EXTRA_ARGS=--with-package-reload --ignore-initial-validation -``` - -## Examples - -This section provides examples to exhibit the use of NSO images. - -### Running the Production Image using Docker CLI - -This example shows how to run the standalone NSO Production Image using the Docker CLI. - -The instructions and CLI examples used in this example are Docker-specific. If you are using a non-Docker container runtime, you will need to: fetch the NSO image from the Cisco software download site, then load and run the image with packages and networking, and finally log in to NSO CLI to run commands. - -If you intend to run multiple images (i.e., both Production and Build), Docker Compose is a tool that simplifies defining and running multi-container Docker applications. See the example ([Running the NSO Images using Docker Compose](containerized-nso.md#sec.example-docker-compose)) below for detailed instructions. - -**Steps** - -Follow the steps below to run the Production Image using Docker CLI: - -1. Start your container engine. -2. Next, load the image and run it. Navigate to the directory where you extracted the base image and load it. This will restore the image and its tag: - -```bash -docker load -i nso-6.4.container-image-prod.linux.x86_64.tar.gz -``` - -3. Start a container from the image. Supply additional arguments to mount the packages and `ncs.conf` as separate volumes ([`-v` flag](https://docs.docker.com/engine/reference/commandline/run/)), and publish ports for networking ([`-p` flag](https://docs.docker.com/engine/reference/commandline/run/)) as needed. The container starts NSO using the `/run-nso.sh` script. To understand how the `ncs.conf` file is used, see [`ncs.conf` File Configuration and Preference](containerized-nso.md#ug.admin_guide.containers.ncs). - -```bash -docker run -itd --name cisco-nso \ --v NSO-vol:/nso \ --v NSO-log-vol:/log \ ---net=host \ --e ADMIN_USERNAME=admin \ --e ADMIN_PASSWORD=admin \ -cisco-nso-prod:6.4 -``` - -{% hint style="warning" %} -**Overriding Environment Variables** - -Overriding basic environment variables (`NCS_CONFIG_DIR`, `NCS_LOG_DIR`, `NCS_RUN_DIR`, etc.) is not supported and therefore should be avoided. Using, for example, the `NCS_CONFIG_DIR` environment variable to mount a configuration directory will result in an error. Instead, to mount your configuration directory, do it appropriately in the correct place, which is under `/nso/etc`. -{% endhint %} - -
- -Examples: Running the Image with and without Named Volumes - -The following examples show how to run the image with and without named volumes. - -**Running without a named volume**: This is the minimal way of running the image but does not provide any persistence when the container is destroyed. - -```bash -docker run -itd --name cisco-nso \ --p 8888:8888 \ --e ADMIN_USERNAME=admin\ --e ADMIN_PASSWORD=admin\ -cisco-nso-prod -``` - -**Running with a single named volume**: This way provides persistence for the NSO mount point with a `NSO-vol` volume. Logs, however, are not persistent. - -```bash - -docker run -itd --name cisco-nso \ --v NSO-vol:/nso \ --p 8888:8888 \ --e ADMIN_USERNAME=admin\ --e ADMIN_PASSWORD=admin\ -cisco-nso-prod -``` - -\ -**Running with two named volumes**: This way provides full persistence for both the NSO and the log mount points. - -```bash -docker run -itd --name cisco-nso \ --v NSO-vol:/nso \ --v NSO-log-vol:/log \ --p 8888:8888 \ --e ADMIN_USERNAME=admin\ --e ADMIN_PASSWORD=admin\ -cisco-nso-prod -``` - -
- -{% hint style="info" %} -**Loading the Packages** - -* Loading the packages by mounting the default load path `/nso/run` as a volume is preferred. You can also load the packages by copying them manually into the `/nso/run/packages` directory in the container. During development, a bind mount of the package directory on the host machine makes it easy to update packages in NSO by simply changing the packages on the host. -* The default load path is configured in the `ncs.conf` file as `$NCS_RUN_DIR/packages`, where `$NCS_RUN_DIR` expands to `/nso/run` in the container. To find the load path, check the `ncs.conf` file in the `/etc/ncs/` directory. - - ```xml - - ${NCS_RUN_DIR}/packages - ${NCS_DIR}/etc/ncs - ... - - ``` -{% endhint %} - -{% hint style="info" %} -**Logging** - -* With the Production Image, use a shared volume to persist data across restarts. If remote (Syslog) logging is used, there is little need to persist logs. If local logging is used, then persistent logging is recommended. -* NSO starts a cron job to handle logrotate of NSO logs by default. i.e., the `CRON_ENABLE` and `LOGROTATE_ENABLE` variables are set to `true` using the `/etc/logrotate.conf` configuration. See the `/etc/ncs/post-ncs-start.d/10-cron-logrotate.sh` script. To set how often the cron job runs, use the crontab file. -{% endhint %} - -4. Finally, log in to NSO CLI to run commands. Open an interactive shell on the running container and access the NSO CLI. - -```bash -docker exec -it cisco-nso bash -# ncs_cli -u admin -admin@ncs> -``` - -You can also use the `docker exec -it cisco-nso ncs_cli -u admin` command to access the CLI from the host's terminal. - -### Upgrading NSO using Docker CLI - -This example describes how to upgrade your NSO to run a newer NSO version in the container. The overall upgrade process is outlined in the steps below. In the example below, NSO is to be upgraded from version 6.3 to 6.4. - -To upgrade your NSO version: - -1. Start a container with the `docker run` command. In the example below, it mounts the `/nso` directory in the container to the `NSO-vol` named volume to persist the data. Another option is using a bind mount of the directory on the host machine. At this point, the `/cdb` directory is empty. - - ```bash - docker run -itd -—name cisco-nso -v NSO-vol:/nso cisco-nso-prod:6.3 - ``` -2. Perform a backup, either by running the `docker exec` command (make sure that the backup is placed somewhere we have mounted) or by creating a tarball of `/data/nso` on the host machine. - - ```bash - docker exec -it cisco-nso ncs-backup - ``` -3. Stop the NSO by issuing the following command, or by stopping the container itself which will run the `ncs stop` command automatically. - - ```bash - docker exec -it cisco-nso ncs --stop - ``` -4. Remove the old NSO. - - ```bash - docker rm -f cisco-nso - ``` -5. Start a new container and mount the `/nso` directory in the container to the `NSO-vol` named volume. This time the `/cdb` folder is not empty, so instead of starting a fresh NSO, an upgrade will be performed. - - ```bash - docker run -itd --name cisco-nso -v NSO-vol:/nso cisco-nso-prod:6.4 - ``` - -At this point, you only have one container that is running the desired version 6.4 and you do not need to uninstall the old NSO. - -### Running the NSO Images using Docker Compose - -This example covers the necessary information to manifest the use of NSO images to compile packages and run NSO. Using Docker Compose is not a requirement, but a simple tool for defining and running a multi-container setup where you want to run both the Production and Build images in an efficient manner. - -#### **Packages** - -The packages used in this example are taken from the [examples.ncs/getting-started/netsim-sshkey](https://github.com/NSO-developer/nso-examples/tree/6.6/getting-started/netsim-sshkey) example: - -* `distkey`: A simple Python + template service package that automates the setup of SSH public key authentication between netsim (ConfD) devices and NSO using a nano service. -* `ne`: A NETCONF NED package representing a netsim network element that implements a configuration subscriber Python application that adds or removes the configured public key, which the netsim (ConfD) network element checks when authenticating public key authentication clients. - -#### **`docker-compose.yaml` - Docker Compose File Example** - -A basic Docker Compose file is shown in the example below. It describes the containers running on a machine: - -* The Production container runs NSO. -* The Build container builds the NSO packages. -* A third `example` container runs the netsim device. - -Note that the packages use a shared volume in this simple example setup. In a more complex production environment, you may want to consider a dedicated redundant volume for your packages. - -``` - version: '1.0' - volumes: - NSO-1-rvol: - - networks: - NSO-1-net: - - services: - NSO-1: - image: cisco-nso-prod:6.4 - container_name: nso1 - profiles: - - prod - environment: - - EXTRA_ARGS=--with-package-reload - - ADMIN_USERNAME=admin - - ADMIN_PASSWORD=admin - networks: - - NSO-1-net - ports: - - "2024:2024" - - "8888:8888" - volumes: - - type: bind - source: /path/to/packages/NSO-1 - target: /nso/run/packages - - type: bind - source: /path/to/log/NSO-1 - target: /log - - type: volume - source: NSO-1-rvol - target: /nso - healthcheck: - test: ncs_cmd -c "wait-start 2" - interval: 5s - retries: 5 - start_period: 10s - timeout: 10s - - BUILD-NSO-PKGS: - image: cisco-nso-build:6.4 - container_name: build-nso-pkgs - network_mode: none - profiles: - - build - volumes: - - type: bind - source: /path/to/packages/NSO-1 - target: /nso/run/packages - - EXAMPLE: - image: cisco-nso-prod:6.4 - container_name: ex-netsim - profiles: - - example - networks: - - NSO-1-net - healthcheck: - test: test -f /nso-run-prod/etc/ncs.conf && ncs-netsim --dir /netsim is-alive ex0 - interval: 5s - retries: 5 - start_period: 10s - timeout: 10s - entrypoint: bash - command: -c 'rm -rf /netsim - && mkdir /netsim - && ncs-netsim --dir /netsim create-network /network-element 1 ex - && PYTHONPATH=/opt/ncs/current/src/ncs/pyapi ncs-netsim --dir - /netsim start - && mkdir -p /nso-run-prod/run/cdb - && echo " - default - admin - admin - admin - " - > /nso-run-prod/run/cdb/init1.xml - && ncs-netsim --dir /netsim ncs-xml-init > - /nso-run-prod/run/cdb/init2.xml - && sed -i.orig -e "s|127.0.0.1|ex-netsim|" - /nso-run-prod/run/cdb/init2.xml - && mkdir -p /nso-run-prod/etc - && sed -i.orig -e "s|| - |" -e "//{n;s|false - | - true|}" defaults/ncs.conf - && sed -i.bak -e "//{n;s| - false|true - |}" defaults/ncs.conf - && sed "//{n;s|false| - true|}" defaults/ncs.conf - > /nso-run-prod/etc/ncs.conf - && mv defaults/ncs.conf.orig defaults/ncs.conf - && tail -f /dev/null' - volumes: - - type: bind - source: /path/to/packages/NSO-1/ne - target: /network-element - - type: volume - source: NSO-1-rvol - target: /nso-run-prod -``` - -
- -Explanation of the Docker Compose File - -A description of noteworthy Compose file items is given below. - -* **`profiles`**: Profiles can be used to group containers in a Compose file, and they work perfectly for the Production, Build, and netsim containers. By adding multiple containers on the same machine (as a developer normally would), you can easily start the Production, Build, and netsim containers using their respective profiles (`prod`, `build`, and `example`). -* **The command used in the netsim example**: Creates a directory called `/netsim` where the netsims will be set up, then starts the netsims, followed by generating two `init.xml` files and editing the `ncs.conf` file for the Production container. Finally, it keeps the container running. If you want this to be more elegant, you need a netsim container image with a script in it that is well-documented. -* **`volumes`**: The Production and Build images are configured intentionally to have the same bind mount with `/path/to/packages/NSO-1` as the source and `/nso/run/packages` as the target. The Production Image mounts both the `/log` and `/nso` directories in the container. The `/log` directory is simply a bind mount, while the `/nso` directory is an actual volume. - - \ - Named volumes are recommended over bind mounts as described by the Docker Volumes documentation. The NSO `/run` directory should therefore be mounted as a named volume. However, you can make the `/run` directory a bind mount as well. - - The Compose file, typically named `docker-compose.yaml`, declares a volume called `NSO-1-rvol`. This is a named volume and will be created automatically by Compose. You can create this volume externally, at which point this volume must be declared as external. If the external volume doesn't exist, the container will not start. - - \ - The `example` netsim container will mount the network element NED in the packages directory. This package should be compiled. Note that the `NSO-1-rvol` volume is used by the `example` container to share the generated `init.xml` and `ncs.conf` files with the NSO Production container. -* **`healthcheck`**: The image comes with its own health check (similar to the one shown here in Compose), and this is how you configure it yourself. The health check for the netsim `example` container checks if the `ncs.conf` file has been generated, and the first Netsim instance started in the container. You could, in theory, start more netsims inside the container. - -
- -#### **Steps** - -Follow the steps below to run the images using Docker Compose: - -1. Start the Build container. This starts the services in the Compose file with the profile `build`. - - ```bash - docker compose --profile build up -d - ``` -2. Copy the packages from the `netsim-sshkey` example and compile them in the NSO Build container. The easiest way to do this is by using the `docker exec` command, which gives more control over what to build and the order of it. You can also do this with a script to make it easier and less verbose. Normally you populate the package directory from the host. Here, we use the packages from an example. - - ```bash - docker exec -it build-nso-pkgs sh -c 'cp -r ${NCS_DIR}/examples.ncs/getting-started \ - /netsim-sshkey/packages ${NCS_RUN_DIR}' - - docker exec -it build-nso-pkgs sh -c 'for f in ${NCS_RUN_DIR}/packages/*/src; \ - do make -C "$f" all || exit 1; done' - ``` -3. Start the netsim container. This outputs the generated `init.xml` and `ncs.conf` files to the NSO Production container. The `--wait` flag instructs to wait until the health check returns healthy. - - ```bash - docker compose --profile example up --wait - ``` -4. Start the NSO Production container. - - ```bash - docker compose --profile prod up --wait - ``` - - \ - At this point, NSO is ready to run the service example to configure the netsim device(s). A bash script (`demo.sh`) that runs the above steps and showcases the `netsim-sshkey` example is given below: - - ``` - #!/bin/bash - set -eu # Abort the script if a command returns with a non-zero exit code or if - # a variable name is dereferenced when the variable hasn't been set - GREEN='\033[0;32m' - PURPLE='\033[0;35m' - NC='\033[0m' # No Color - - printf "${GREEN}##### Reset the container setup\n${NC}"; - docker compose --profile build down - docker compose --profile example down -v - docker compose --profile prod down -v - rm -rf ./packages/NSO-1/* ./log/NSO-1/* - - printf "${GREEN}##### Start the build container used for building the NSO NED - and service packages\n${NC}" - docker compose --profile build up -d - - printf "${GREEN}##### Get the packages\n${NC}" - printf "${PURPLE}##### NOTE: Normally you populate the package directory from the host. - Here, we use packages from an NSO example\n${NC}" - docker exec -it build-nso-pkgs sh -c 'cp -r - ${NCS_DIR}/examples.ncs/getting-started/netsim-sshkey/packages ${NCS_RUN_DIR}' - - printf "${GREEN}##### Build the packages\n${NC}" - docker exec -it build-nso-pkgs sh -c 'for f in ${NCS_RUN_DIR}/packages/*/src; - do make -C "$f" all || exit 1; done' - - printf "${GREEN}##### Start the simulated device container and setup the example\n${NC}" - docker compose --profile example up --wait - - printf "${GREEN}##### Start the NSO prod container\n${NC}" - docker compose --profile prod up --wait - - printf "${GREEN}##### Showcase the netsim-sshkey example from NSO on the prod container\n${NC}" - if [[ $# -eq 0 ]] ; then # Ask for input only if no argument was passed to this script - printf "${PURPLE}##### Press any key to continue or ctrl-c to exit\n${NC}" - read -n 1 -s -r - fi - docker exec -it nso1 sh -c 'sed -i.orig -e "s/make/#make/" - ${NCS_DIR}/examples.ncs/getting-started/netsim-sshkey/showcase.sh' - docker exec -it nso1 sh -c 'cd ${NCS_RUN_DIR}; - ${NCS_DIR}/examples.ncs/getting-started/netsim-sshkey/showcase.sh 1' - ``` - -### Upgrading NSO using Docker Compose - -This example describes how to upgrade NSO when using Docker Compose. - -#### **Upgrade to a New Minor or Major Version** - -To upgrade to a new minor or major version, for example, from 6.3 to 6.4, follow the steps below: - -1. Change the image version in the Compose file to the new version, here 6.4. -2. Run the `docker compose up --profile build -d` command to start the Build container with the new image. -3. Compile the packages using the Build container. - - ```bash - docker exec -it build-nso-pkgs sh -c 'for f in - ${NCS_RUN_DIR}/packages/*/src;do make -C "$f" all || exit 1; done' - ``` -4. Run the `docker compose up --profile prod --wait` command to start the Production container with the new packages that were just compiled. - -#### **Upgrade to a New Maintenance Version** - -To upgrade to a new maintenance release version, for example, 6.4.1, follow the steps below: - -1. Change the image version in the Compose file to the new version, here 6.4.1. -2. Run the `docker compose up --profile prod --wait` command. - - Upgrading in this way does not require a recompile. Docker detects changes and upgrades the image in the container to the new version. diff --git a/administration/installation-and-deployment/deployment/deployment-example.md b/administration/installation-and-deployment/deployment/deployment-example.md deleted file mode 100644 index 5b089dce..00000000 --- a/administration/installation-and-deployment/deployment/deployment-example.md +++ /dev/null @@ -1,348 +0,0 @@ ---- -description: Understand NSO deployment with an example setup. ---- - -# Deployment Example - -This section shows examples of a typical deployment for a highly available (HA) setup. A reference to an example implementation of the `tailf-hcc` layer-2 upgrade deployment scenario described here, check the NSO example set under [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc). The example covers the following topics: - -* Installation of NSO on all nodes in an HA setup -* Initial configuration of NSO on all nodes -* HA failover -* Upgrading NSO on all nodes in the HA cluster -* Upgrading NSO packages on all nodes in the HA cluster - -The deployment examples use both the legacy rule-based and recommended HA Raft setup. See [High Availability](../../management/high-availability.md) for HA details. The HA Raft deployment consists of three nodes running NSO and a node managing them, while the rule-based HA deployment uses only two nodes. - -Based on the Raft consensus algorithm, the HA Raft version provides the best fault tolerance, performance, and security and is therefore recommended. - -For the HA Raft setup, the NSO nodes `paris.fra`, `london.eng`, and `berlin.ger` nodes make up a cluster of one leader and two followers. - -

The HA Raft Deployment Network

- -For the rule-based HA setup, the NSO nodes `paris` and `london` make up one HA pair — one primary and one secondary. - -

The Rule-Based HA Deployment Network

- -HA is usually not optional for a deployment. Data resides in CDB, a RAM database with a disk-based journal for persistence. Both HA variants can be set up to avoid the need for manual intervention in a failure scenario, where HA Raft does the best job of keeping the cluster up. See [High Availability](../../management/high-availability.md) for details. - -## Initial NSO Installation - -An NSO system installation on the NSO nodes is recommended for deployments. For System Installation details, see the [System Install](../system-install.md) steps. - -In this container-based example, Docker Compose uses a `Dockerfile` to build the container image and install NSO on multiple nodes, here containers. A shell script uses an SSH client to access the NSO nodes from the manager node to demonstrate HA failover and, as an alternative, a Python script that implements SSH and RESTCONF clients. - -* An `admin` user is created on the NSO nodes. Password-less `sudo` access is set up to enable the `tailf-hcc` server to run the `ip` command. The manager's SSH client uses public key authentication, while the RESTCONF client uses a token to authenticate with the NSO nodes. - - The example creates two packages using the `ncs-make-package` command: `dummy` and `inert`. A third package, `tailf-hcc`, provides VIPs that point to the current HA leader/primary node. -* The packages are compressed into a `tar.gz` format for easier distribution, but that is not a requirement. - -{% hint style="info" %} -While this deployment example uses containers, it is intended as a generic deployment guide. For details on running NSO in a container, such as Docker, see [Containerized NSO](../containerized-nso.md). -{% endhint %} - -This example uses a minimal Red Hat UBI distribution for hosting NSO with the following added packages: - -* NSO's basic dependency requirements are fulfilled by adding the Java Runtime Environment (JRE), OpenSSH, and OpenSSL packages. -* The OpenSSH server is used for shell access and secure copy to the NSO Linux host for NSO version upgrade purposes. The NSO built-in SSH server provides CLI and NETCONF access to NSO. -* The NSO services require Python. -* To fulfill the `tailf-hcc` server dependencies, the `iproute2` utilities and `sudo` packages are installed. See [Dependencies](../../management/high-availability.md#ug.ha.hcc.deps) (in the section [Tailf HCC Package](../../management/high-availability.md#ug.ha.hcc)) for details on dependencies. -* The `rsyslog` package enables storing an NSO log file from several NSO logs locally and forwarding some logs to the manager. -* The `arp` command from the `net-tools` and `iputils` (`ping`) packages have been added for demonstration purposes. - -The steps in the list below are performed as `root`. Docker Compose will build the container images, i.e., create the NSO installation as `root`. - -The `admin` user will only need `root` access to run the `ip` command when `tailf-hcc` adds the Layer 2 VIP address to the leader/primary node interface. - -The initialization steps are also performed as `root` for the nodes that make up the HA cluster: - -* Create the `ncsadmin` and `ncsoper` Linux user groups. -* Create and add the `admin` and `oper` Linux users to their respective groups. -* Perform a system installation of NSO that runs NSO as the `admin` user. -* The `admin` user is granted access to run the `ip` command from the `vipctl` script as `root` using the `sudo` command as required by the `tailf-hcc` package. -* The `cmdwrapper` NSO program gets access to run the scripts executed by the `generate-token` action for generating RESTCONF authentication tokens as the current NSO user. -* Password authentication is set up for the read-only `oper` user for use with NSO only, which is intended for WebUI access. -* The `root` user is set up for Linux shell access only. -* The NSO installer, `tailf-hcc` package, application YANG modules, scripts for generating and authenticating RESTCONF tokens, and scripts for running the demo are all available to the NSO and manager containers. -* `admin` user permissions are set for the NSO directories and files created by the system install, as well as for the `root`, `admin`, and `oper` home directories. -* The `ncs.crypto_keys` are generated and distributed to all nodes.\ - \ - **Note**: The `ncs.crypto_keys` file is highly sensitive. It contains the encryption keys for all encrypted CDB data, which often includes passwords for various entities, such as login credentials to managed devices.\ - \ - **Note**: In an NSO System Install setup, not only the TLS certificates (HA Raft) or shared token (rule-based HA) need to match between the HA cluster nodes, but also the configuration for encrypted strings, by default stored in `/etc/ncs/ncs.crypto_keys`, needs to match between the nodes in the HA cluster. For rule-based HA, the tokens configured on the secondary nodes are overwritten with the encrypted token of type `aes-256-cfb-128-encrypted-string` from the primary node when the secondary connects to the primary. If there is a mismatch between the encrypted-string configuration on the nodes, NSO will not decrypt the HA token to match the token presented. As a result, the primary node denies the secondary node access the next time the HA connection needs to be re-established with a "Token mismatch, secondary is not allowed" error. -* For HA Raft, TLS certificates are generated for all nodes. -* The initial NSO configuration, `ncs.conf`, is updated and in sync (identical) on the nodes. -* The SSH servers are configured to allow only SSH public key authentication (no password). The `oper` user can use password authentication with the WebUI but has read-only NSO access. -* The `oper` user is denied access to the Linux shell. -* The `admin` user can access the Linux shell and NSO CLI using public key authentication. -* New keys for all users are distributed to the HA cluster nodes and the manager node when the HA cluster is initialized. -* The OpenSSH server and the NSO built-in SSH server use the same private and public key pairs located under `~/.ssh/id_ed25519`, while the manager public key is stored in the `~/.ssh/authorized_keys` file for both NSO nodes. -* Host keys are generated for all nodes to allow the NSO built-in SSH and OpenSSH servers to authenticate the server to the client.\ - \ - Each HA cluster node has its own unique SSH host keys stored under `${NCS_CONFIG_DIR}/ssh_host_ed25519_key`. The SSH client(s), here the manager, has the keys for all nodes in the cluster paired with the node's hostname and the VIP address in its `/root/.ssh/known_hosts` file.\ - \ - The host keys, like those used for client authentication, are generated each time the HA cluster nodes are initialized. The host keys are distributed to the manager and nodes in the HA cluster before the NSO built-in SSH and OpenSSH servers are started on the nodes. -* As NSO runs in containers, the environment variables are set to point to the system install directories in the Docker Compose `.env` file. -* NSO runs as the non-root `admin` user and, therefore, the NSO system installation is done using the `./nso-${VERSION}.linux.${ARCH}.installer.bin --system-install --run-as-user admin --ignore-init-scripts` options. By default, the NSO installation start script will create a `systemd` system service to run NSO as the `admin` user (default is the `root` user) when NSO is started using the `systemctl start ncs` command.\ - \ - However, this example uses the `--ignore-init-scripts` option to skip installing `systemd` scripts as it runs in a container that does not support `systemd`.\ - \ - The environment variables are copied to a `.pam_environment` file so the `root` and `admin` users can set the required environment variables when those users access the shell via SSH.\ - \ - The `/etc/systemd/system/ncs.service` `systemd` service script is installed as part of the NSO system install, if not using the `--ignore-init-scripts` option, and it can be customized if you would like to use it to start NSO. The script may provide what you need and can be a starting point. -* The OpenSSH `sshd` and `rsyslog` daemons are started. -* The packages from the package store are added to the `${NCS_RUN_DIR}/packages` directory before finishing the initialization part in the `root` context. -* The NSO smart licensing token is set. - -## The `ncs.conf` Configuration - -* The NSO IPC socket is configured in `ncs.conf` to only listen to localhost 127.0.0.1 connections, which is the default setting.\ - \ - By default, the clients connecting to the NSO IPC socket are considered trusted, i.e., no authentication is required, and the use of 127.0.0.1 with the `/ncs-config/ncs-ipc-address` IP address in `ncs.conf` to prevent remote access. See [Security Considerations](deployment-example.md#ug.admin_guide.deployment.security) and [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) in Manual Pages for more details. -* `/ncs-config/aaa/pam` is set to enable PAM to authenticate users as recommended. All remote access to NSO must now be done using the NSO host's privileges. See [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) in Manual Pages for details. -* Depending on your Linux distribution, you may have to change the `/ncs-config/aaa/pam/service` setting. The default value is `common-auth`. Check the file `/etc/pam.d/common-auth` and make sure it fits your needs. See [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) in Manual Pages for details.\ - \ - Alternatively, or as a complement to the PAM authentication, users can be stored in the NSO CDB database or authenticated externally. See [Authentication](../../management/aaa-infrastructure.md#ug.aaa.authentication) for details. -* RESTCONF token authentication under `/ncs-config/aaa/external-validation` is enabled using a `token_auth.sh` script that was added earlier together with a `generate_token.sh` script. See [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) in Manual Pages for details.\ - \ - The scripts allow users to generate a token for RESTCONF authentication through, for example, the NSO CLI and NETCONF interfaces that use SSH authentication or the Web interface. - - The token provided to the user is added to a simple YANG list of tokens where the list key is the username. -* The token list is stored in the NSO CDB operational data store and is only accessible from the node's local MAAPI and CDB APIs. See the HA Raft and rule-based HA `upgrade-l2/manager-etc/yang/token.yang` file in the examples. -* The NSO web server HTTPS interface should be enabled under `/ncs-config/webui`, along with `/ncs-config/webui/match-host-name = true` and `/ncs-config/webui/server-name` set to the hostname of the node, following security best practice. If the server needs to serve multiple domains or IP addresses, additional `server-alias` values can be configured. See [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) in Manual Pages for details. - - **Note**: The SSL certificates that NSO generates are self-signed: - - ```bash - $ openssl x509 -in /etc/ncs/ssl/cert/host.cert -text -noout - Certificate: - Data: - Version: 1 (0x0) - Serial Number: 2 (0x2) - Signature Algorithm: sha256WithRSAEncryption - Issuer: C=US, ST=California, O=Internet Widgits Pty Ltd, CN=John Smith - Validity - Not Before: Dec 18 11:17:50 2015 GMT - Not After : Dec 15 11:17:50 2025 GMT - Subject: C=US, ST=California, O=Internet Widgits Pty Ltd - Subject Public Key Info: - ....... - ``` - - Thus, if this is a production environment and the JSON-RPC and RESTCONF interfaces using the web server are not used solely for internal purposes, the self-signed certificate must be replaced with a properly signed certificate. See [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) in Manual Pages under `/ncs-config/webui/transport/ssl/cert-file` and `/ncs-config/restconf/transport/ssl/certFile` for more details. -* Disable `/ncs-config/webui/cgi` unless needed. -* The NSO SSH CLI login is enabled under `/ncs-config/cli/ssh/enabled`. See [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) in Manual Pages for details. -* The NSO CLI style is set to C-style, and the CLI prompt is modified to include the hostname under `/ncs-config/cli/prompt`. See [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) in Manual Pages for details. - - ```xml - \u@nso-\H> - \u@nso-\H% - - \u@nso-\H# - \u@nso-\H(\m)# - ``` -* NSO HA Raft is enabled under `/ncs-config/ha-raft`, and the rule-based HA under `/ncs-config/ha`. See [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) in Manual Pages for details. -* Depending on your provisioned applications, you may want to turn `/ncs-config/rollback/enabled` off. Rollbacks do not work well with nano service reactive FASTMAP applications or if maximum transaction performance is a goal. If your application performs classical NSO provisioning, the recommendation is to enable rollbacks. Otherwise not. See [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) in Manual Pages for details. - -## The `aaa_init.xml` Configuration - -The NSO System Install places an AAA `aaa_init.xml` file in the `$NCS_RUN_DIR/cdb` directory. Compared to a Local Install for development, no users are defined for authentication in the `aaa_init.xml` file, and PAM is enabled for authentication. NACM rules for controlling NSO access are defined in the file for users belonging to a `ncsadmin` user group and read-only access for a `ncsoper` user group. As seen in the previous sections, this example creates Linux `root`, `admin`, and `oper` users, as well as the `ncsadmin` and `ncsoper` Linux user groups. - -PAM authenticates the users using SSH public key authentication without a passphrase for NSO CLI and NETCONF login. Password authentication is used for the `oper` user intended for NSO WebUI login and token authentication for RESTCONF login. - -Before the NSO daemon is running, and there are no existing CDB files, the default AAA configuration in the `aaa_init.xml` is used. It is restrictive and is used for this demo with only a minor addition to allow the oper user to generate a token for RESTCONF authentication. - -The NSO authorization system is group-based; thus, for the rules to apply to a specific user, the user must be a member of the group to which the restrictions apply. PAM performs the authentication, while the NSO NACM rules do the authorization. - -* Adding the `admin` user to the `ncsadmin` group and the `oper` user to the limited `ncsoper` group will ensure that the two users get properly authorized with NSO. -* Not adding the `root` user to any group matching the NACM groups results in zero access, as no NACM rule will match, and the default in the `aaa_init.xml` file is to deny all access. - -The NSO NACM functionality is based on the [Network Configuration Access Control Model](https://datatracker.ietf.org/doc/html/rfc8341) IETF RFC 8341 with NSO extensions augmented by `tailf-acm.yang`. See [AAA infrastructure](../../management/aaa-infrastructure.md), for more details. - -The manager in this example logs into the different NSO hosts using the Linux user login credentials. This scheme has many advantages, mainly because all audit logs on the NSO hosts will show who did what and when. Therefore, the common bad practice of having a shared `admin` Linux user and NSO local user with a shared password is not recommended. - -{% hint style="info" %} -The default `aaa_init.xml` file provided with the NSO system installation must not be used as-is in a deployment without reviewing and verifying that every NACM rule in the file matches the desired authorization level. -{% endhint %} - -## The High Availability and VIP Configuration - -This example sets up one HA cluster using HA Raft or rule-based HA with the `tailf-hcc` server to manage virtual IP addresses. See [NSO Rule-based HA](../../management/high-availability.md) and [Tail-f HCC Package](../../management/high-availability.md#ug.ha.hcc) for details. - -The NSO HA, together with the `tailf-hcc` package, provides three features: - -* All CDB data is replicated from the leader/primary to the follower/secondary nodes. -* If the leader/primary fails, a follower/secondary takes over and starts to act as leader/primary. This is how HA Raft works and how the rule-based HA variant of this example is configured to handle failover automatically. -* At failover, `tailf-hcc` sets up a virtual alias IP address on the leader/primary node only and uses gratuitous ARP packets to update all nodes in the network with the new mapping to the leader/primary node. - -Nodes in other networks can be updated using the `tailf-hcc` layer-3 BGP functionality or a load balancer. See the `load-balancer`and `hcc`examples in the NSO example set under [examples.ncs/high-availability](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability). - -See the NSO example set under [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc) for a reference to an HA Raft and rule-based HA `tailf-hcc` Layer 3 BGP examples. - -The HA Raft and rule-based HA upgrade-l2 examples also demonstrate HA failover, upgrading the NSO version on all nodes, and upgrading NSO packages on all nodes. - -## Global Settings and Timeouts - -Depending on your installation, e.g., the size and speed of the managed devices and the characteristics of your service applications, some default values of NSO may have to be tweaked, particularly some of the timeouts. - -* Device timeouts. NSO has connect, read, and write timeouts for traffic between NSO and the managed devices. The default value may not be sufficient if devices/nodes are slow to commit, while some are sometimes slow to deliver their full configuration. Adjust timeouts under `/devices/global-settings` accordingly. -* Service code timeouts. Some service applications can sometimes be slow. Adjusting the `/services/global-settings/service-callback-timeout` configuration might be applicable depending on the applications. However, the best practice is to change the timeout per service from the service code using the Java `ServiceContext.setTimeout` function or the Python `data_set_timeout` function. - -There are quite a few different global settings for NSO. The two mentioned above often need to be changed. - -## Cisco Smart Licensing - -NSO uses Cisco Smart Licensing, which is described in detail in [Cisco Smart Licensing](../../management/system-management/cisco-smart-licensing.md). After registering your NSO instance(s), and receiving a token, following steps 1-6 as described in the [Create a License Registration Token](../../management/system-management/cisco-smart-licensing.md#d5e2927) section of Cisco Smart Licensing, enter a token from your Cisco Smart Software Manager account on each host. Use the same token for all instances and script entering the token as part of the initial NSO configuration or from the management node: - -```bash -admin@nso-paris# license smart register idtoken YzY2Yj... -admin@nso-london# license smart register idtoken YzY2Yj... -``` - -{% hint style="info" %} -The Cisco Smart Licensing CLI command is present only in the Cisco Style CLI, which is the default CLI for this setup. -{% endhint %} - -## Log Management - -### Log Rotate - -The NSO system installations performed on the nodes in the HA cluster also install defaults for **logrotate**. Inspect `/etc/logrotate.d/ncs` and ensure that the settings are what you want. Note that the NSO error logs, i.e., the files `/var/log/ncs/ncserr.log*`, are internally rotated by NSO and must not be rotated by `logrotate`. - -### Syslog - -For the HA Raft and rule-based HA upgrade-l2 examples, see the reference from the `README` in the [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc) example directory; the examples integrate with `rsyslog` to log the `ncs`, `developer`, `upgrade`, `audit`, `netconf`, `snmp`, and `webui-access` logs to syslog with `facility` set to `daemon` in `ncs.conf`. - -`rsyslogd` on the nodes in the HA cluster is configured to write the daemon facility logs to `/var/log/daemon.log`, and forward the daemon facility logs with the severity `info` or higher to the manager node's `/var/log/ha-cluster.log` syslog. - -### Audit Network Log and NED Traces - -Use the audit-network-log for recording southbound traffic towards devices. Enable by setting `/ncs-config/logs/audit-network-log/enabled` and `/ncs-config/logs/audit-network-log/file/enabled` to true in `$NCS_CONFIG_DIR/ncs.conf`, See [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) in Manual Pages for more information. - -NED trace logs are a crucial tool for debugging NSO installations and not recommended for deployment. These logs are very verbose and for debugging only. Do not enable these logs in production. - -Note that the NED logs include everything, even potentially sensitive data is logged. No filtering is done. The NED trace logs are controlled through the CLI under: `/device/global-settings/trace`. It is also possible to control the NED trace on a per-device basis under `/devices/device[name='x']/trace`. - -There are three different settings for trace output. For various historical reasons, the setting that makes the most sense depends on the device type. - -* For all CLI NEDs, use the `raw` setting. -* For all ConfD and netsim-based NETCONF devices, use the pretty setting. This is because ConfD sends the NETCONF XML unformatted, while `pretty` means that the XML is formatted. -* For Juniper devices, use the `raw` setting. Juniper devices sometimes send broken XML that cannot be formatted appropriately. However, their XML payload is already indented and formatted. -* For generic NED devices - depending on the level of trace support in the NED itself, use either `pretty` or `raw`. -* For SNMP-based devices, use the `pretty` setting. - -Thus, it is usually not good enough to control the NED trace from `/devices/global-settings/trace`. - -### Python Logs - -While there is a global log for, for example, compilation errors in `/var/log/ncs/ncs-python-vm.log`, logs from user application packages are written to separate files for each package, and the log file naming is `ncs-python-vm-`_`pkg_name`_`.log`. The level of logging from Python code is controlled on a per package basis. See [Debugging of Python packages](../../../development/core-concepts/nso-virtual-machines/nso-python-vm.md#debugging-of-python-packages) for more details. - -### Java Logs - -User application Java logs are written to `/var/log/ncs/ncs-java-vm.log`. The level of logging from Java code is controlled per Java package. See [Logging](../../../development/core-concepts/nso-virtual-machines/nso-java-vm.md#logging) in Java VM for more details. - -### Internal NSO Log - -The internal NSO log resides at `/var/log/ncs/ncserr.*`. The log is written in a binary format. To view the internal error log, run the following command: - -```bash - $ ncs --printlog /var/log/ncs/ncserr.log.1 -``` - -## Monitoring the Installation - -All large-scale deployments employ monitoring systems. There are plenty of good tools to choose from, open source and commercial. All good monitoring tools can script (using various protocols) what should be monitored. It is recommended that a special read-only Linux user without shell access be set up like the `oper` user earlier in this chapter. A few commonly used checks include: - -* At startup, check that NSO has been started using the `$NCS_DIR/bin/ncs_cmd -c "wait-start 2"` command. -* Use the `ssh` command to verify SSH access to the NSO host and NSO CLI. -* Check disk usage using, for example, the `df` utility. -* For example, use **curl** or the Python requests library to verify that the RESTCONF API is accessible. -* Check that the NETCONF API is accessible using, for example, the `$NCS_DIR/bin/netconf-console` tool with a `hello` message. -* Verify the NSO version using, for example, the `$NCS_DIR/bin/ncs --version` or RESTCONF `/restconf/data/tailf-ncs-monitoring:ncs-state/version`. -* Check if HA is enabled using, for example, RESTCONF `/restconf/data/tailf-ncs-monitoring:ncs-state/ha`. - -### Alarms - -RESTCONF can be used to view the NSO alarm table and subscribe to alarm notifications. NSO alarms are not events. Whenever an NSO alarm is created, a RESTCONF notification and SNMP trap are also sent, assuming that you have a RESTCONF client registered with the alarm stream or configured a proper SNMP target. Some alarms, like the rule-based HA `ha-secondary-down` alarm, require the intervention of an operator. Thus, a monitoring tool should also fetch the NSO alarm list. - -```bash -$ curl -ik -H "X-Auth-Token: TsZTNwJZoYWBYhOPuOaMC6l41CyX1+oDaasYqQZqqok=" \ -https://paris:8888/restconf/data/tailf-ncs-alarms:alarms -``` - -Or subscribe to the `ncs-alarms` RESTCONF notification stream. - -### Metric - Counters, Gauges, and Rate of Change Gauges - -NSO metric has different contexts all containing different counters, gauges, and rate of change gauges. There is a `sysadmin`, a `developer` and a `debug` context. Note that only the `sysadmin` context is enabled by default, as it is designed to be lightweight. Consult the YANG module `tailf-ncs-metric.yang` to learn the details of the different contexts. - -### **Counters** - -You may read counters by e.g. CLI, as in this example: - -```bash -admin@ncs# show metric sysadmin counter session cli-total -metric sysadmin counter session cli-total 1 -``` - -### **Gauges** - -You may read gauges by e.g. CLI, as in this example: - -```bash -admin@ncs# show metric sysadmin gauge session cli-open -metric sysadmin gauge session cli-open 1 -``` - -### **Rate of Change Gauges** - -You may read rate of change gauges by e.g. CLI, as in this example: - -```bash -admin@ncs# show metric sysadmin gauge-rate session cli-open -NAME RATE -------------- -1m 0.0 -5m 0.2 -15m 0.066 -``` - -## Security Considerations - -This section covers security considerations for this example. See [Secure Deployment Considerations](secure-deployment.md) for a general description. - -The presented configuration enables the built-in web server for the WebUI and RESTCONF interfaces. It is paramount for security that you only enable HTTPS access with `/ncs-config/webui/match-host-name` and `/ncs-config/webui/server-name` properly set. - -The AAA setup described so far in this deployment document is the recommended AAA setup. To reiterate: - -* Have all users that need access to NSO authenticated through Linux PAM. This may then be through `/etc/passwd`. Avoid storing users in CDB. -* Given the default NACM authorization rules, you should have three different types of users on the system. - * Users with shell access are members of the `ncsadmin` Linux group and are considered fully trusted because they have full access to the system. - * Users without shell access who are members of the `ncsadmin` Linux group have full access to the network. They have access to the NSO SSH shell and can execute RESTCONF calls, access the NSO CLI, make configuration changes, etc. However, they cannot manipulate backups or perform system upgrades unless such actions are added to by NSO applications. - * Users without shell access who are members of the `ncsoper` Linux group have read-only access. They can access the NSO SSH shell, read data using RESTCONF calls, etc. However, they cannot change the configuration, manipulate backups, and perform system upgrades. - -If you have more fine-grained authorization requirements than read-write and read-only, additional Linux groups can be created, and the NACM rules can be updated accordingly. See [The `aaa_init.xml` Configuration](deployment-example.md#ug.admin_guide.deployment.aaa) from earlier in this chapter on how the reference example implements users, groups, and NACM rules to achieve the above. - -The default `aaa_init.xml` file must not be used as-is before reviewing and verifying that every NACM rule in the file matches the desired authorization level. - -For a detailed discussion of the configuration of authorization rules through NACM, see [AAA infrastructure](../../management/aaa-infrastructure.md), particularly the section [Authorization](../../management/aaa-infrastructure.md#ug.aaa.authorization). - -A considerably more complex scenario is when users require shell access to the host but are either untrusted or should not have any access to NSO at all. NSO listens to a so-called IPC socket configured through `/ncs-config/ncs-ipc-address`. This socket is typically limited to local connections and defaults to `127.0.0.1:4569` for security. The socket multiplexes several different access methods to NSO. - -The main security-related point is that no AAA checks are performed on this socket. If you have access to the socket, you also have complete access to all of NSO. - -To drive this point home, when you invoke the `ncs_cli` command, a small C program that connects to the socket and tells NSO who you are, NSO assumes that authentication has already been performed. There is even a documented flag `--noaaa`, which tells NSO to skip all NACM rule checks for this session. - -You must protect the socket to prevent untrusted Linux shell users from accessing the NSO instance using this method. This is done by using a file in the Linux file system. The file `/etc/ncs/ipc_access` gets created and populated with random data at install time. Enable `/ncs-config/ncs-ipc-access-check/enabled` in `ncs.conf` and ensure that trusted users can read the `/etc/ncs/ipc_access` file, for example, by changing group access to the file. See [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) in Manual Pages for details. - -```bash -$ cat /etc/ncs/ipc_access -cat: /etc/ncs/ipc_access: Permission denied -$ sudo chown root:ncsadmin /etc/ncs/ipc_access -$ sudo chmod g+r /etc/ncs/ipc_access -$ ls -lat /etc/ncs/ipc_access -$ cat /etc/ncs/ipc_access -....... -``` - -For an HA setup, HA Raft is based on the Raft consensus algorithm and provides the best fault tolerance, performance, and security. It is therefore recommended over the legacy rule-based HA variant. The `raft-upgrade-l2` project, referenced from the NSO example set under [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc), together with this Deployment Example section, describes a reference implementation. See [NSO HA Raft](../../management/high-availability.md#ug.ha.raft) for more HA Raft details. diff --git a/administration/installation-and-deployment/deployment/develop-and-deploy-a-nano-service.md b/administration/installation-and-deployment/deployment/develop-and-deploy-a-nano-service.md deleted file mode 100644 index 38d21775..00000000 --- a/administration/installation-and-deployment/deployment/develop-and-deploy-a-nano-service.md +++ /dev/null @@ -1,427 +0,0 @@ ---- -description: Develop and deploy a nano service using a guided example. ---- - -# Develop and Deploy a Nano Service - -This section shows how to develop and deploy a simple NSO nano service for managing the provisioning of SSH public keys for authentication. For more details on nano services, see [Nano Services for Staged Provisioning](../../../development/core-concepts/nano-services.md) in Development. The example showcasing development is available under [examples.ncs/getting-started/netsim-sshkey](https://github.com/NSO-developer/nso-examples/tree/6.6/getting-started/netsim-sshkey). In addition, there is a reference from the `README` in the example's directory to the deployment version of the example. - -## Development - -

The Development Host Topology

- -After installing NSO with the [Local Install](../local-install.md) option, development often begins with either retrieving an existing YANG model representing what the managed network element (a virtual or physical device, such as a router) can do or constructing a new YANG model that at least covers the configuration of interest to an NSO service. To enable NSO service development, the network element's YANG model can be used with NSO's netsim tool that uses ConfD (Configuration Daemon) to simulate the network elements and their management interfaces like NETCONF. Read more about netsim in [Network Simulator](../../../operation-and-usage/operations/network-simulator-netsim.md). - -The simple network element YANG model used for this example is available under `packages/ne/src/yang/ssh-authkey.yang`. The `ssh-authkey.yang` model implements a list of SSH public keys for identifying a user. The list of keys augments a list of users in the ConfD built-in `tailf-aaa.yang` module that ConfD uses to authenticate users. - -```yang -module ssh-authkey { - yang-version 1.1; - namespace "http://example.com/ssh-authkey"; - prefix sa; - - import tailf-common { - prefix tailf; - } - - import tailf-aaa { - prefix aaa; - } - - description - "List of SSH authorized public keys"; - - revision 2023-02-02 { - description - "Initial revision."; - } - - augment "/aaa:aaa/aaa:authentication/aaa:users/aaa:user" { - list authkey { - key pubkey-data; - leaf pubkey-data { - type string; - } - } - } -} -``` - -On the network element, a Python application subscribes to ConfD to be notified of configuration changes to the user's public keys and updates the user's authorized\_keys file accordingly. See `packages/ne/netsim/ssh-authkey.py` for details. - -The first step is to create an NSO package from the network element YANG model. Since NSO will use NETCONF over SSH to communicate with the device, the package will be a NETCONF NED. The package can be created using the `ncs-make-package` command or the NETCONF NED builder tool. The `ncs-make-package` command is typically used when the YANG models used by the network element are available. Hence, the packages/ne package for this example was generated using the `ncs-make-package` command. - -As the `ssh-authkey.yang` model augments the users list in the ConfD built-in `tailf-aaa.yang` model, NSO needs a representation of that YANG model too to build the NED. However, the service will only configure the user's public keys, so only a subset of the `tailf-aaa.yang` model that only includes the user list is sufficient. To compare, see the `packages/ne/src/yang/tailf-aaa.yang` in the example vs. the network element's version under `$NCS_DIR/netsim/confd/src/confd/aaa/tailf-aaa.yang`. - -Now that the network element package is defined, next up is the service package, beginning with finding out what steps are required for NSO to authenticate with the network element using SSH public key authentication: - -1. First, generate private and public keys using, for example, the `ssh-keygen` OpenSSH authentication key utility. -2. Distribute the public keys to the ConfD-enabled network element's list of authorized keys. -3. Configure NSO to use public key authentication with the network element. -4. Finally, test the public key authentication by connecting NSO with the network element. - -The outline above indicates that the service will benefit from implementing several smaller (nano) steps: - -* The first step only generates private and public key files with no configuration. Thus, the first step should be implemented by an action before the second step runs, not as part of the second step transaction `create()` callback code configuring the network elements. The `create()` callback runs multiple times, for example, for service configuration changes, re-deploy, or commit dry-run. Therefore, generating keys should only happen when creating the service instance. -* The third step cannot be executed before the second step is complete, as NSO cannot use the public key for authenticating with the network element before the network element has it in its list of authorized keys. -* The fourth step uses the NSO built-in `connect()` action and should run after the third step finishes. - -What configuration input do the above steps need? - -* The name of the network element that will authenticate a user with an SSH public key. -* The name of the local NSO user that maps to the remote network element user the public key authenticates. -* The name of the remote network element user. -* A passphrase is used for encrypting the private key, guarding its privacy. The passphrase should be encrypted when storing it in the CDB, just like any other password. -* The name of the NSO authentication group to configure for public-key authentication with the NSO-managed network element. - -A service YANG model that implements the above configuration: - -```yang - container pubkey-dist { - list key-auth { - key "ne-name local-user"; - - uses ncs:nano-plan-data; - uses ncs:service-data; - ncs:servicepoint "distkey-servicepoint"; - - leaf ne-name { - type leafref { - path "/ncs:devices/ncs:device/ncs:name"; - } - } - leaf local-user { - type leafref { - path "/ncs:devices/ncs:authgroups/ncs:group/ncs:umap/ncs:local-user"; - require-instance false; - } - } - leaf remote-name { - type leafref { - path "/ncs:devices/ncs:authgroups/ncs:group/ncs:umap/ncs:remote-name"; - require-instance false; - } - mandatory true; - } - leaf authgroup-name { - type leafref { - path "/ncs:devices/ncs:authgroups/ncs:group/ncs:name"; - require-instance false; - } - mandatory true; - } - leaf passphrase { - // Leave unset for no passphrase - tailf:suppress-echo true; - type tailf:aes-256-cfb-128-encrypted-string { - length "10..max" { - error-message "The passphrase must be at least 10 characters long"; - } - pattern ".*[a-z]+.*" { - error-message "The passphrase must have at least one lower case alpha"; - } - pattern ".*[A-Z]+.*" { - error-message "The passphrase must have at least one upper case alpha"; - } - pattern ".*[0-9]+.*" { - error-message "The passphrase must have at least one digit"; - } - pattern ".*[<>~;:!@#/$%^&*=-]+.*" { - error-message "The passphrase must have at least one of these" + - " symbols: [<>~;:!@#/$%^&*=-]+"; - } - pattern ".* .*" { - modifier invert-match; - error-message "The passphrase must have no spaces"; - } - } - } - ... - } - } -``` - -For details on the YANG statements used by the YANG model, such as `leaf`, `container`, `list`, `leafref`, `mandatory`, `length`, `pattern`, etc., see the [IETF RFC 7950](https://www.rfc-editor.org/rfc/rfc7950) that documents the YANG 1.1 Data Modeling Language. The `tailf:xyz` are YANG extension statements documented by [tailf\_yang\_extensions(5)](../../../resources/man/tailf_yang_extensions.5.md) in Manual Pages. - -The service configuration is implemented in YANG by a `key-auth` list where the network element and local user names are the list keys. In addition, the list has a `distkey-servicepoint` service point YANG extension statement to enable the list parameters used by the Python service callbacks that this example implements. Finally, the used `service-data` and `nano-plan-data` groupings add the common definitions for a service and the plan data needed when the service is a nano service. - -For the nano service YANG part, an NSO YANG nano service behavior tree extension that references a plan outline extension implements the above steps for setting up SSH public key authentication with a network element: - -``` - ncs:plan-outline distkey-plan { - description "Plan for distributing a public key"; - ncs:component-type "dk:ne" { - ncs:state "ncs:init"; - ncs:state "dk:generated" { - ncs:create { - // Request the generate-keys action - ncs:post-action-node "$SERVICE" { - ncs:action-name "generate-keys"; - ncs:result-expr "result = 'true'"; - ncs:sync; - } - } - ncs:delete { - // Request the delete-keys action - ncs:post-action-node "$SERVICE" { - ncs:action-name "delete-keys"; - ncs:result-expr "result = 'true'"; - } - } - } - ncs:state "dk:distributed" { - ncs:create { - // Invoke a Python program to distribute the authorized public key to - // the network element - ncs:nano-callback; - ncs:force-commit; - } - } - ncs:state "dk:configured" { - ncs:create { - // Invoke a Python program that in turn invokes a service template to - // configure NSO to use public key authentication with the network - // element - ncs:nano-callback; - // Request the connect action to test the public key authentication - ncs:post-action-node "/ncs:devices/device[name=$NE-NAME]" { - ncs:action-name "connect"; - ncs:result-expr "result = 'true'"; - } - } - } - ncs:state "ncs:ready"; - } - } - ncs:service-behavior-tree distkey-servicepoint { - description "One component per distkey behavior tree"; - ncs:plan-outline-ref "dk:distkey-plan"; - ncs:selector { - // The network element name used with this component - ncs:variable "NE-NAME" { - ncs:value-expr "current()/ne-name"; - } - // The unique component name - ncs:variable "NAME" { - ncs:value-expr "concat(current()/ne-name, '-', current()/local-user)"; - } - // Component for setting up public key authentication - ncs:create-component "$NAME" { - ncs:component-type-ref "dk:ne"; - } - } - } -``` - -The nano `service-behavior-tree` for the service point creates a nano service component for each list entry in the `key-auth` list. The last connection verification step of the nano service, the `connected` state, uses the `NE-NAME` variable. The `NAME` variable concatenates the `ne-name` and `local-user` keys from the `key-auth` list to create a unique nano service component name. - -The only step that requires both a create and delete part is the `generated` state action that generates the SSH keys. If a user deletes a service instance and another network element does not currently use the generated keys, this deletes the keys too. NSO will revert the configuration automatically as part of the FASTMAP algorithm. Hence, the service list instances also need actions for generating and deleting keys. - -```yang - container pubkey-dist { - list key-auth { - key "ne-name local-user"; - ... - action generate-keys { - tailf:actionpoint generate-keys; - output { - leaf result { - type boolean; - } - } - } - action delete-keys { - tailf:actionpoint delete-keys; - output { - leaf result { - type boolean; - } - } - } - } - } -``` - -The actions have no input statements, as the input is the configuration in the service instance list entry. - -The `generated` state uses the `ncs:sync` statement to ensure that the keys exist before the `distributed` state runs. Similarly, the `distributed` state uses the `force-commit` statement to commit the configuration to the NSO CDB and the network elements before the `configured` state runs. - -See the `packages/distkey/src/yang/distkey.yang` YANG model for the nano service behavior tree, plan outline, and service configuration implementation. - -Next, handling the key generation, distributing keys to the network element, and configuring NSO to authenticate using the keys with the network element requires some code, here written in Python, implemented by the `packages/distkey/python/distkey/distkey-app.py` script application. - -The Python script application defines a Python `DistKeyApp` class specified in the `packages/distkey/package-meta-data.xml` file that NSO starts in a Python thread. This Python class inherits `ncs.application.Application` and implements the `setup()` and `teardown()` methods. The `setup()` method registers the nano service `create()` callbacks and the action handlers for generating and deleting the key files. Using the nano service state to separate the two nano service `create()` callbacks for the distribution and NSO configuration of keys, only one Python class, the `DistKeyServiceCallbacks` class, is needed to implement them. - -```python -class DistKeyApp(ncs.application.Application): - def setup(self): - # Nano service callbacks require a registration for a service point, - # component, and state, as specified in the corresponding data model - # and plan outline. - self.register_nano_service('distkey-servicepoint', # Service point - 'dk:ne', # Component - 'dk:distributed', # State - DistKeyServiceCallbacks) - self.register_nano_service('distkey-servicepoint', # Service point - 'dk:ne', # Component - 'dk:configured', # State - DistKeyServiceCallbacks) - - # Side effect action that uses ssh-keygen to create the keyfiles - self.register_action('generate-keys', GenerateActionHandler) - # Action to delete the keys created by the generate keys action - self.register_action('delete-keys', DeleteActionHandler) - - def teardown(self): - self.log.info('DistKeyApp FINISHED') -``` - -The action for generating keys calls the OpenSSH `ssh-keygen` command to generate the private and public key files. Calling `ssh-keygen` is kept out of the service `create()` callback to avoid the key generation running multiple times, for example, for service changes, re-deploy, or dry-run commits. Also, NSO encrypts the passphrase used when generating the keys for added security, see the YANG model, so the Python code decrypts it before using it with the `ssh-keygen` command. - -```python -class GenerateActionHandler(Action): - @Action.action - def cb_action(self, uinfo, name, keypath, ainput, aoutput, trans): - '''Action callback''' - service = ncs.maagic.get_node(trans, keypath) - # Install the crypto keys used to decrypt the service passphrase leaf - # as input to the key generation. - with ncs.maapi.Maapi() as maapi: - _maapi.install_crypto_keys(maapi.msock) - # Decrypt the passphrase leaf for use when generating the keys - encrypted_passphrase = service.passphrase - decrypted_passphrase = _ncs.decrypt(str(encrypted_passphrase)) - aoutput = True - # If it does not exist already, generate a private and public key - if os.path.isfile(f'./{service.local_user}_ed25519') == False: - result = subprocess.run(['ssh-keygen', '-N', - f'{decrypted_passphrase}', '-t', 'ed25519', - '-f', f'./{service.local_user}_ed25519'], - stdout=subprocess.PIPE, check=True, - encoding='utf-8') - if "has been saved" not in result.stdout: - aoutput = False -``` - -The `DeleteActionHandler` action deletes the key files if no more network elements use the user's keys: - -```python -class DeleteActionHandler(Action): - @Action.action - def cb_action(self, uinfo, name, keypath, ainput, aoutput, trans): - '''Action callback''' - service = ncs.maagic.get_node(trans, keypath) - # Only delete the key files if no more network elements use this - # user's keys - cur = trans.cursor('/pubkey-dist/key-auth') - remove_key = True - while True: - try: - value = next(cur) - if value[0] != service.ne_name and value[1] == service.local_user: - remove_key = False - break - except StopIteration: - break - aoutput = True - if remove_key is True: - try: - os.remove(f'./{service.local_user}_ed25519.pub') - os.remove(f'./{service.local_user}_ed25519') - except OSError as e: - if e.errno != errno.ENOENT: - aoutput = False -``` - -The Python class for the nano service `create()` callbacks handles both the distribution and NSO configuration of the keys. The `dk:distributed` state `create()` callback code adds the public key data to the network element's list of authorized keys. For the `create()` call for the `dk:configured` state, a template is used to configure NSO to use public key authentication with the network element. The template can be called directly from the nano service, but in this case, it needs to be called from the Python code to input the current working directory to the template: - -```python -class DistKeyServiceCallbacks(NanoService): - @NanoService.create - def cb_nano_create(self, tctx, root, service, plan, component, state, - proplist, component_proplist): - '''Nano service create callback''' - if state == 'dk:distributed': - # Distribute the public key to the network element's authorized - # keys list - with open(f'./{service.local_user}_ed25519.pub', 'r') as f: - pubkey_data = f.read() - config = root.devices.device[service.ne_name].config - users = config.aaa.authentication.users - users.user[service.local_user].authkey.create(pubkey_data) - elif state == 'dk:configured': - # Configure NSO to use a public key for authentication with - # the network element - template_vars = ncs.template.Variables() - template_vars.add('CWD', os.getcwd()) - template = ncs.template.Template(service) - template.apply('distkey-configured', template_vars) -``` - -The template to configure NSO to use public key authentication with the network element is available under `packages/distkey/templates/distkey-configured.xml`: - -```xml - - - - - {authgroup-name} - - {local-user} - {remote-name} - - - - {$CWD}/{local-user}_ed25519 - {passphrase} - - - - - - - - {ne-name} - {authgroup-name} - - -} -``` - -The example uses three scripts to showcase the nano service: - -* A shell script, `showcase.sh`, which uses the `ncs_cli` program to run CLI commands via the NSO IPC port. -* A Python script, `showcase-rc.sh`, which uses the `requests` package for RESTCONF edit operations and receiving event notifications. -* A Python script that uses NSO MAAPI, `showcase-maapi.sh`, via the NSO IPC port. - -The `ncs_cli` program identifies itself with NSO as the `admin` user without authentication, and the RESTCONF client uses plain HTTP and basic user password authentication. All three scripts demonstrate the service by generating keys, distributing the public key, and configuring NSO for public key authentication with the network elements. To run the example, see the instructions in the `README` file of the example. - -## Deployment - -See the `README` in the `netsim-sshkey` example's directory for a reference to an NSO system installation in a container deployment variant. - -

The Deployment Container Topology

- -The deployment variant differs from the development example by: - -* Installing NSO with a system installation for deployment instead of a local installation suitable for development -* Addressing NSO security by running NSO as the `admin` user and authenticating using a public key and token. -* Rotating NSO logs to avoid running out of disk space -* Installing the `distkey` service package and `ne` NED package at startup -* The NSO CLI showcase script uses SSH with public key authentication instead of the **ncs\_cli** program over unsecured IPC -* There is no Python MAAPI showcase script. Use RESTCONF over HTTPS with Python instead of Python MAAPI over unsecured IPC. -* Having NSO and the network elements (simulated by the ConfD subscriber application) run in separate containers -* NSO is either pre-installed in the NSO production container image or installed in a generic Linux container. - -The deployment example sets up a minimal production installation where the NSO process runs as the `admin` OS user, relying on PAM authentication for the `admin` and `oper` NSO users. The `admin` user is authenticated over SSH using a public key for CLI and NETCONF access and over RESTCONF HTTPS using a token. The read-only `oper` user uses password authentication. The `oper` user can access the NSO WebUI over HTTPS port 443 from the container host. - -A modified version of the NSO configuration file `ncs.conf` from the example running with a local install NSO is located in the `$NCS_CONFIG_DIR` (`/etc/ncs`) directory. The `packages`, `ncs-cdb`, `state`, and `scripts` directories are now under the `$NCS_RUN_DIR` (`/var/opt/ncs`) directory. The log directory is now the `$NCS_LOG_DIR` (`/var/log/ncs`) directory. Finally, the `$NCS_DIR` variable points to `/opt/ncs/current`. - -Two scripts showcase the nano service: - -* A shell script that runs NSO CLI commands over SSH. -* A Python script that uses the `requests` package to perform edit operations and receive event notifications. - -As with the development version, both scripts will demo the service by generating keys, distributing the public key, and configuring NSO for public key authentication with the network elements. - -To run the example and for more details, see the instructions in the `README` file of the [examples.ncs/getting-started/netsim-sshkey](https://github.com/NSO-developer/nso-examples/tree/6.6/getting-started/netsim-sshkey) deployment example. diff --git a/administration/installation-and-deployment/deployment/secure-deployment.md b/administration/installation-and-deployment/deployment/secure-deployment.md deleted file mode 100644 index 04ba84fe..00000000 --- a/administration/installation-and-deployment/deployment/secure-deployment.md +++ /dev/null @@ -1,199 +0,0 @@ ---- -description: Security features to consider for NSO deployment. ---- - -# Secure Deployment - -When deploying NSO in production environments, security should be a primary consideration. This section guides the NSO features available for securing your NSO deployment. - -## Development vs. Production Deployment - -NSO installations can be configured for development or production use, with significantly different security implications. - -### Production Installation - -* Use the NSO Installer with the `--system-install` option for production deployments. - * The `--local-install` option should only be used for development environments. - * Use the NSO Installer `--run-as-user ` option to run NSO as a non-root user. -* Never use `ncs.conf` files from NSO distribution examples in production. - * Evaluate and customize the default `ncs.conf` file provided with a system installation to meet your specific security requirements. - -### Key Configuration Differences - -The default `ncs.conf` for production installations differs from the development default `ncs.conf` in several critical security areas: - -#### Encryption Keys - -* Production (system) installations use external key management where `ncs.conf` points to `${NCS_CONFIG_DIR}/ncs.crypto_keys` using the `${NCS_DIR}/bin/ncs_crypto_keys` command to retrieve them. -* Development installations include the encryption keys directly in `ncs.conf`. - -#### SSH Configuration - -* Production restricts SSH host key algorithms to `ssh-ed25519` only. -* Development allows multiple algorithms for compatibility. - -#### Authentication - -* Production disables local authentication by default, using PAM with `system-auth`. -* Development enables local authentication and uses PAM with `common-auth`. -* Production includes password expiration warnings. - -#### Network Interfaces - -* Production disables CLI SSH, HTTP WebUI, and NETCONF SSH by default. -* Development enables these interfaces for convenience. -* Production enables restricted-file-access for CLI. - -See [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) for all available options to configure the NSO daemon. - -## Eliminating Root Access - -Running NSO with minimal privileges is a fundamental security best practice: - -* Use the NSO installer `--run-as-user User` option to run NSO as a non-root user. -* Some files are installed with elevated privileges - refer to the [ncs-installer(1)](../../../resources/man/ncs-installer.1.md#system-installation) man page under the `--run-as-user User` option for details. -* The NSO production container runs NSO from a [non-root user](../containerized-nso.md#nso-runs-from-a-non-root-user). -* If the CLI is used and we want to create CLI commands that run executables, we may want to modify the permissions of the `$NCS_DIR/lib/ncs/lib/confd-*/priv/cmdptywrapper` program.\ - To be able to run an executable as root or a specific user, we need to make `cmdptywrapper` `setuid` `root`, i.e.: - - 1. `# chown root cmdptywrapper` - 2. `# chmod u+s cmdptywrapper` - - Failing that, all programs will be executed as the user running the `ncs` daemon. Consequently, if that user is the `root`, we do not have to perform the `chmod` operations above. The same applies to executables run via actions, but then we may want to modify the permissions of the `$NCS_DIR/lib/ncs/lib/confd-*/priv/cmdwrapper` program instead: - - 1. `# chown root cmdwrapper` - 2. `# chmod u+s cmdwrapper` -* The deployment variant referenced in the README file of the [examples.ncs/getting-started/netsim-sshkey](https://github.com/NSO-developer/nso-examples/tree/6.6/getting-started/netsim-sshkey) example provides a native and NSO production container based example. - -## Authentication, Authorization, and Accounting (AAA) - -### PAM Authentication - -PAM (Pluggable Authentication Modules) is the recommended authentication method for NSO: - -* Group assignments based on the OS group database `/etc/group`. -* Default NACM (Network Configuration Access Control Module) settings provide two groups: - * `ncsadmin`: unlimited access rights. - * `ncsoper`: minimal access rights (read-only). - -See [PAM](../../management/aaa-infrastructure.md#ug.aaa.pam) for details. - -### Customizing AAA Configuration - -When customizing the default `aaa_init.xml` configuration: - -* Exclude credentials unless local authentication is explicitly enabled. -* Never use default passwords. -* Carefully consider which groups can modify NACM rules. -* Tailor NACM settings for user groups based on the principle of least privilege. - -See [AAA Infrastructure](../../management/aaa-infrastructure.md) for details. - -### Additional Authentication Methods - -* CLI and NETCONF: SSH public key authentication. -* RESTCONF: Token, JWT, LDAP, or TACACS+ authentication. -* WebUI: HTTPS (TLS) with JSON-RPC SSO (Single Sign-On). - -{% hint style="info" %} -Disable unused interfaces in `ncs.conf` to reduce the attack surface. -{% endhint %} - -See [Authentication](../../management/aaa-infrastructure.md#ug.aaa.authentication) for details. - -## Securing IPC Access - -Inter-Process Communication (IPC) security is crucial for safeguarding NSO's extensibility SDK API communications. Since the IPC socket allows full control of the system, it is important to ensure that only trusted or authorized clients can connect. See [Restricting Access to the IPC Socket](../../advanced-topics/ipc-connection.md#restricting-access-to-the-ipc-socket). - -Examples of programs that connect using IPC sockets: - -* Remote commands, such as `ncs --reload`. -* MAAPI, CDB, DP, event notification API clients. -* The `ncs_cli` program. -* The `ncs_cmd` and `ncs_load` programs. - -### Default Security - -* Only local connections to IPC sockets are allowed by default. -* TCP sockets with no authentication. - -### Best Practices - -* Use Unix sockets for authenticating the client based on the UID of the other end of the socket connection. - * Root and the user NSO runs from always have access. - * If using TCP sockets, configure NSO to use access checks with a pre-shared key. - * If enabling non-localhost IPC over TCP sockets, implement encryption. - -See [Authenticating IPC Access](../../management/aaa-infrastructure.md#authenticating-ipc-access) for details. - -## Southbound Interface Security - -Secure communication with managed devices: - -* Use [Cisco-provided NEDs](../../management/ned-administration.md) when possible. -* Refer to the [examples.ncs/getting-started/netsim-sshkey](https://github.com/NSO-developer/nso-examples/tree/6.6/getting-started/netsim-sshkey) README, which references a deployment variant of the example for SSH key update patterns using nano services. - -## Cryptographic Key Management - -### Hashing Algorithms - -* Set the `ncs.conf` `/ncs-config/crypt-hash/algorithm` to SHA-512 for password hashing. - * Used by the `ianach:crypt-hash` type for secure password storage. - -### Encryption Keys - -* Generate new encryption keys before or at startup. -* Replace or rotate keys generated by the NSO installer. - * Rotate keys periodically. -* Store keys securely (default location: `/etc/ncs/ncs.crypto_keys`). -* The `ncs.crypto_keys` file contains the highly sensitive encryption keys for all encrypted CDB data. - -See [Cryptographic Keys](../../advanced-topics/cryptographic-keys.md) for details. - -## Rate Limiting and Resource Protection - -Implement various limiting mechanisms to prevent resource exhaustion: - -### NSO Configuration Limits - -NSO can be configured with some limits from `ncs.conf`: - -* `/ncs-config/session-limits`: Limit concurrent sessions. -* `/ncs-config/transaction-limits`: Limit concurrent transactions. -* `/ncs-config/parser-limits`: Limit XML data parsing. -* `/ncs-config/webui/transport/unauthenticated-message-limit`: Limit unauthenticated message size. -* `/ncs-config/webui/rate-limiting`: Limit JSON-RPC requests per hour. - -### External Rate Limiting - -For additional protection, implement rate limiting at the network level using tools like Linux `iptables`. - -## High Availability Security - -When deploying NSO in [HA (High Availability)](../../management/high-availability.md) configurations: - -* RAFT HA: - * Uses encrypted TLS with mutual X.509 authentication. -* Rule-based HA: - * Unencrypted communication. - * Shared token for authentication between HA group nodes. - -{% hint style="info" %} -Encrypted strings for all encrypted CDB data, default stored in `/etc/ncs/ncs.crypto_keys`, must be identical across nodes -{% endhint %} - -## Compliance Reporting - -NSO provides comprehensive [compliance reporting](../../../operation-and-usage/operations/compliance-reporting.md) capabilities: - -* Track user actions - "Who has done what?" -* Verify network configuration compliance. -* Generate audit reports for regulatory requirements. - -## FIPS Mode - -For enhanced security and regulatory compliance: - -* FIPS mode restricts NSO to use only FIPS 140-3 validated cryptographic modules. -* Enable with the `--fips-install` option during [installation](../system-install.md). -* Required for certain government and regulated industry deployments. diff --git a/administration/installation-and-deployment/development-to-production-deployment/README.md b/administration/installation-and-deployment/development-to-production-deployment/README.md deleted file mode 100644 index 82f5cc47..00000000 --- a/administration/installation-and-deployment/development-to-production-deployment/README.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -description: Deploy NSO from development to production. ---- - -# Development to Production Deployment - diff --git a/administration/installation-and-deployment/local-install.md b/administration/installation-and-deployment/local-install.md deleted file mode 100644 index 6575bad2..00000000 --- a/administration/installation-and-deployment/local-install.md +++ /dev/null @@ -1,619 +0,0 @@ ---- -description: >- - Install NSO for non-production use, such as for development and training - purposes. ---- - -# Local Install - -## Installation Steps - -Complete the following activities in the given order to perform a Local Install of NSO. - -
Prepare1. Fulfill System Requirements
2. Download Installer/NEDs
3. Unpack the Installer
Install4. Run the Installer
Finalize5. Set Environment Variables
6. Runtime Directory Creation
7. Generate License Token
- -{% hint style="info" %} -**Mode of Install** - -NSO Local Install can be installed in **standard mode** or in [**FIPS**](https://www.nist.gov/itl/publications-0/federal-information-processing-standards-fips)**-compliant mode**. Standard mode install supports a broader set of cryptographic algorithms, while the FIPS mode install restricts NSO to use only FIPS 140-3-validated cryptographic modules and algorithms for enhanced/regulated security and compliance. Use FIPS mode only in environments that require compliance with specific security standards, especially in U.S. federal agencies or regulated industries. For all other use cases, install NSO in standard mode. - -\* FIPS: Federal Information Processing Standards -{% endhint %} - -### Step 1 - Fulfill System Requirements - -Start by setting up your system to install and run NSO. - -To install NSO: - -1. Fulfill at least the primary requirements. -2. If you intend to build and run NSO examples, you also need to install additional applications listed under Additional Requirements. - -{% hint style="warning" %} -Where requirements list a specific or higher version, there always exists a (small) possibility that a higher version introduces breaking changes. If in doubt whether the higher version is fully backwards compatible, always use the specific version. -{% endhint %} - -
- -Primary Requirements - -Primary requirements to do a Local Install include: - -* A system running Linux or macOS on either the `x86_64` or `ARM64` architecture for development. For [FIPS](https://www.nist.gov/itl/publications-0/federal-information-processing-standards-fips) mode, OS FIPS compliance may be required depending on your specific requirements. -* GNU libc 2.24 or higher. -* Java JRE 17 or higher. Used by Cisco Smart Licensing. -* Required and included with many Linux/macOS distributions: - * `tar` command. Unpack the installer. - * `gzip` command. Unpack the installer. - * `ssh-keygen` command. Generate SSH host key. - * `openssl` command. Generate self-signed certificates for HTTPS. - * `find` command. Used to find out if all required libraries are available. - * `which` command. Used by the NSO package manager. - * `libpam.so.0`. Pluggable Authentication Module library. - * `libexpat.so.1`. EXtensible Markup Language parsing library. - * `libz.so.1` version 1.2.7.1 or higher. Data compression library. - -
- -
- -Additional Requirements - -Additional requirements to, for example, build and run NSO examples/services include: - -* Java JDK 17 or higher. -* Ant 1.9.8 or higher. -* Python 3.10 or higher. -* Python Setuptools is required to build the Python API. -* Often installed using the Python package installer pip: - * Python Paramiko 2.2 or higher. To use netconf-console. - * Python requests. Used by the RESTCONF demo scripts. -* `xsltproc` command. Used by the `support/ned-make-package-meta-data` command to generate the `package-meta-data.xml` file. -* One of the following web browsers is required for NSO GUI capabilities. The version must be supported by the vendor at the time of release. - * Safari - * Mozilla Firefox - * Microsoft Edge - * Google Chrome -* OpenSSH client applications. For example, the `ssh` and `scp` commands. - -
- -
- -FIPS Mode Entropy Requirements - -The following applies if you are running a container-based setup of your FIPS install: - -In containerized environments (e.g., Docker) that run on older Linux kernels (e.g., Ubuntu 18.04), `/dev/random` may block if the system’s entropy pool is low. This can lead to delays or hangs in FIPS mode, as cryptographic operations require high-quality randomness. - -To avoid this: - -* Prefer newer kernels (e.g., Ubuntu 22.04 or later), where entropy handling is improved to mitigate the issue. -* Or, install an entropy daemon like Haveged on the Docker host to help maintain sufficient entropy. - -Check available entropy on the host system with: - -```bash -cat /proc/sys/kernel/random/entropy_avail -``` - -A value of 256 or higher is generally considered safe. Reference: [Oracle blog post](https://blogs.oracle.com/linux/post/entropyavail-256-is-good-enough-for-everyone). - -
- -### Step 2 - Download the Installer and NEDs - -To download the Cisco NSO installer and example NEDs: - -1. Go to the Cisco's official [Software Download](https://software.cisco.com/download/home) site. -2. Search for the product "Network Services Orchestrator" and select the desired version. -3. There are two versions of the NSO installer, i.e. for macOS and Linux systems. Download the desired installer. - -
- -Identifying the Installer - -You need to know your system specifications (Operating System and CPU architecture) in order to choose the appropriate NSO installer. - -NSO is delivered as an OS/CPU-specific signed self-extractable archive. The signed archive file has the pattern `nso-VERSION.OS.ARCH.signed.bin` that after signature verification extracts the `nso-VERSION.OS.ARCH.installer.bin` archive file, where: - -* `VERSION` is the NSO version to install. -* `OS` is the Operating System (`linux` for all Linux distributions and `darwin` for macOS). -* `ARCH` is the CPU architecture, for example`x86_64`. - -
- -### Step 3 - Unpack the Installer - -If your downloaded file is a `signed.bin` file, it means that it has been digitally signed by Cisco, and upon execution, you will verify the signature and unpack the `installer.bin`. - -If you only have `installer.bin`, skip to the next step. - -To unpack the installer: - -1. In the terminal, list the binaries in the directory where you downloaded the installer, for example: - - ```bash - cd ~/Downloads - ls -l nso*.bin - -rw-r--r--@ 1 user staff 199M Dec 15 11:45 nso-6.0.darwin.x86_64.installer.bin - -rw-r--r--@ 1 user staff 199M Dec 15 11:45 nso-6.0.darwin.x86_64.signed.bin - ``` -2. Use the `sh` command to run the `signed.bin` to verify the certificate and extract the installer binary and other files. An example output is shown below. - - ```bash - sh nso-6.0.darwin.x86_64.signed.bin - # Output - Unpacking... - Verifying signature... - Downloading CA certificate from http://www.cisco.com/security/pki/certs/crcam2.cer ... - Successfully downloaded and verified crcam2.cer. - Downloading SubCA certificate from http://www.cisco.com/security/pki/certs/innerspace.cer ... - Successfully downloaded and verified innerspace.cer. - Successfully verified root, subca and end-entity certificate chain. - Successfully fetched a public key from tailf.cer. - Successfully verified the signature of nso-6.0.darwin.x86_64.installer.bin using tailf.cer - ``` -3. List the files to check if extraction was successful. - - ```bash - ls -l - # Output - -rw-r--r-- 1 user staff 1.8K Nov 29 06:05 README.signature - -rw-r--r-- 1 user staff 12K Nov 29 06:05 cisco_x509_verify_release.py - -rwxr-xr-x 1 user staff 199M Nov 29 05:55 nso-6.0.darwin.x86_64.installer.bin - -rw-r--r-- 1 user staff 256B Nov 29 06:05 nso-6.0.darwin.x86_64.installer.bin.signature - -rwxr-xr-x@ 1 user staff 199M Dec 15 11:45 nso-6.0.darwin.x86_64.signed.bin - -rw-r--r-- 1 user staff 1.4K Nov 29 06:05 tailf.cer - ``` - -
- -Description of Unpacked Files - -The following contents are unpacked: - -* `nso-VERSION.OS.ARCH.installer.bin`: The NSO installer. -* `nso-VERSION.OS.ARCH.installer.bin.signature`: Signature generated for the NSO image. -* `tailf.cer`: An enclosed Cisco signed x.509 end-entity certificate containing the public key that is used to verify the signature. -* `README.signature`: File with further details on the unpacked content and steps on how to run the signature verification program. To manually verify the signature, refer to the steps in this file. -* `cisco_x509_verify_release.py`: Python program that can be used to verify the 3-tier x.509 certificate chain and signature. -* Multiple `.tar.gz` files: Bundled packages, extending the base NSO functionality. -* Multiple `.tar.gz.signature` files: Digital signatures for the bundled packages. - -Since NSO version 6.3, a few additional NSO packages are included. They contain the following platform tools: - -* HCC -* Observability Exporter - -- Phased Provisioning -- Resource Manager - -For platform tools documentation, refer to the individual package's `README` file or to the [online documentation](https://nso-docs.cisco.com/resources). - -**NED packages** - -The NED packages that are available with the NSO installation are NetSim-based example NEDs. These NEDs are used for NSO examples only. - -Fetch the latest production-grade NEDs from [Cisco Software Download](https://software.cisco.com/download/home) using the URLs provided on your NED license certificates. - -**Manual pages** - -The installation program unpacks the NSO manual pages from the documentation archive in `$NCS_DIR/man`. `ncsrc` makes an addition to `$MANPATH`, allowing you to use the `man` command to view them. The manual pages are available in PDF format and from the online documentation located on [NCS man-pages, Volume 1](../../resources/man/README.md) in Manual Pages. - -Following is a list of a few of the installed manual pages: - -* `ncs(1)`: Command to start and control the NSO daemon. -* `ncsc(1)`: NSO Yang compiler. -* `ncs_cli(1)`: Frontend to the NSO CLI engine. -* `ncs-netsim(1)`: Command to create and manipulate a simulated network. -* `ncs-setup(1)`: Command to create an initial NSO setup. -* `ncs.conf`: NSO daemon configuration file format. - -For example, to view the manual page describing the NSO configuration file, you should type: - -```bash -$ man ncs.conf -``` - -Apart from the manual pages, extensive information about command-line options can be obtained by running `ncs` and `ncsc` with the `--help` (abbreviated `-h`) flag. - -```bash -$ ncs --help -``` - -```bash -$ ncsc --help -``` - -**Installer help** - -Run the `sh nso-VERSION.darwin.x86_64.installer.bin --help` command to view additional help on running binaries. More details can be found in the [ncs-installer(1)](../../resources/man/ncs-installer.1.md) Manual Page included with NSO. - -Notice the two options for `--local-install` or `--system-install`. An example output is shown below. - -```bash -sh nso-6.0.darwin.x86_64.installer.bin --help - -# Output -This is the NCS installation script. -Usage: ./nso-6.0.darwin.x86_64.installer.bin [--local-install] LocalInstallDir -Installs NCS in the LocalInstallDir directory only. -This is convenient for test and development purposes. -Usage: ./nso-6.0.darwin.x86_64.installer.bin --system-install -[--install-dir InstallDir] -[--config-dir ConfigDir] [--run-dir RunDir] [--log-dir LogDir] -[--run-as-user User] [--keep-ncs-setup] [--non-interactive] - -Does a system install of NCS, suitable for deployment. -Static files are installed in InstallDir/ncs-. -The first time --system-install is used, the ConfigDir, -RunDir, and LogDir directories are also created and -populated for config files, run-time state files, and log files, -respectively, and an init script for start of NCS at system boot -and user profile scripts are installed. Defaults are: - -InstallDir - /opt/ncs -ConfigDir - /etc/ncs -RunDir - /var/opt/ncs -LogDir - /var/log/ncs - -By default, the system install will run NCS as the root user. -If the --run-as-user option is given, the system install will -instead run NCS as the given user. The user will be created if -it does not already exist. -If the --non-interactive option is given, the installer will -proceed with potentially disruptive changes (e.g. modifying or -removing existing files) without asking for confirmation. -``` - -
- -### Step 4 - Run the Installer - -Local Install of NSO software is performed in a single user-specified directory, for example in your `$HOME` directory. - -{% hint style="success" %} -It is always recommended to install NSO in a directory named as the version of the release, for example, if the version being installed is `6.1`, the directory should be `~/nso-6.1`. -{% endhint %} - -To run the installer: - -1. Navigate to your Install Directory. -2. Run the command given below to install NSO in your Install Directory. The `--local-install` parameter is optional. At this point, you can choose to install NSO in standard mode or in FIPS mode. - -{% tabs %} -{% tab title="Standard Local Install" %} -The standard mode is the regular NSO install and is suitable for most installations. FIPS is disabled in this mode. - -For standard NSO install, run the installer as below: - -```bash -$ sh nso-VERSION.OS.ARCH.installer.bin $HOME/ncs-VERSION --local-install -``` - -An example output is shown below: - -{% code title="Example: Standard Local Install" %} -```bash -sh nso-6.0.darwin.x86_64.installer.bin --local-install ~/nso-6.0 - -# Output -INFO Using temporary directory /var/folders/90/n5sbctr922336_ -0jrzhb54400000gn/T//ncs_installer.93831 to stage NCS installation bundle -INFO Unpacked ncs-6.0 in /Users/user/nso-6.0 -INFO Found and unpacked corresponding DOCUMENTATION_PACKAGE -INFO Found and unpacked corresponding EXAMPLE_PACKAGE -INFO Found and unpacked corresponding JAVA_PACKAGE -INFO Generating default SSH hostkey (this may take some time) -INFO SSH hostkey generated -INFO Environment set-up generated in /Users/user/nso-6.0/ncsrc -INFO NSO installation script finished -INFO Found and unpacked corresponding NETSIM_PACKAGE -INFO NCS installation complete -``` -{% endcode %} -{% endtab %} - -{% tab title="FIPS Local Install" %} -FIPS mode creates a FIPS-compliant NSO install. - -FIPS mode should only be used for deployments that are subject to strict compliance regulations as the cryptographic functions are then confined to the CiscoSSL FIPS 140-3 module library. - -For FIPS-compliant NSO install, run the installer with the additional `--fips-install` flag. Afterwards, verify FIPS in `ncs.conf`. - -```bash -$ sh nso-VERSION.OS.ARCH.installer.bin $HOME/ncs-VERSION --local-install --fips-install -``` - -{% hint style="info" %} -**NSO Configuration for FIPS** - -Note the following as part of FIPS-specific configuration/install: - -1. The `ncs.conf` file is automatically configured to enable FIPS by setting the following flag: - -```xml - - true - -``` - -2. Additional environment variables (`NCS_OPENSSL_CONF_INCLUDE`, `NCS_OPENSSL_CONF`, `NCS_OPENSSL_MODULES`) are configured in `ncsrc` for FIPS compliance. -3. The default `crypto.so` is overwritten at install for FIPS compliance. - -Additionally, note that: - -* As certain algorithms typically available with CiscoSSL are not included in the FIPS 140-3 validated module (and therefore disabled in FIPS mode), you need to configure NSO to use only the algorithms and cryptographic suites available through the CiscoSSL FIPS 140-3 object module. -* With FIPS, NSO signals the NEDs to operate in FIPS mode using Bouncy Castle FIPS libraries for Java-based components, ensuring compliance with FIPS 140-3. To support this, NED packages may also require upgrading, as older versions — particularly SSH-based NEDs — often lack the necessary FIPS signaling or Bouncy Castle support required for cryptographic compliance. -* Configure SSH keys in `ncs.conf` and `init.xml`. -{% endhint %} -{% endtab %} -{% endtabs %} - -### Step 5 - Set Environment Variables - -The installation program creates a shell script file named `ncsrc` in each NSO installation, which sets the environment variables. - -To set the environment variables: - -1. Source the `ncsrc` file to get the environment variables settings in your shell. You may want to add this sourcing command to your login sequence, such as `.bashrc`. - - For `csh/tcsh` users, there is a `ncsrc.tcsh` file with `csh/tcsh` syntax. The example below assumes that you are using `bash`, other versions of `/bin/sh` may require that you use `.` instead of `source`. - - ```bash - $ source $HOME/ncs-VERSION/ncsrc - ``` -2. Most users add source `~/nso-x.x/ncsrc` (where `x.x` is the NSO version) to their `~/.bash_profile`, but you can simply do it manually when you want it. Once it has been sourced, you have access to all the NSO executable commands, which start with `ncs`. - - ```bash - ncs {TAB} {TAB} - - # Output - ncs ncs-maapi ncs-project ncs-start-python-vm ncs_cmd - ncs-backup ncs-make-package ncs-setup ncs-uninstall ncs_conf_tool - ncs-collect ncs-netsim ncs-start-java-vm ncs_cli - - ncs_load - ncsc - ncs_crypto_keys-tech-report - ``` - -### Step 6 - Create Runtime Directory - -NSO needs a deployment/runtime directory where the database files, logs, etc. are stored. An empty default directory can be created using the `ncs-setup` command. - -To create a Runtime Directory: - -1. Create a Runtime Directory for NSO by running the following command. In this case, we assume that the directory is `$HOME/ncs-run`. - - ```bash - $ ncs-setup --dest $HOME/ncs-run - ``` -2. Start the NSO daemon `ncs`. - - ```bash - $ cd $HOME/ncs-run - $ ncs - ``` - -
- -Runtime vs. Installation Directory - -A common misunderstanding is that there exists a dependency between the Runtime Directory and the Installation Directory. This is not true. For example, say that you have two NSO local installations `path/to/nso-6.4` and `path/to/nso-6.4.1`. The following sequence runs `nso-6.4` but uses an example and configuration from `nso-6.4.1`. - -```bash - $ cd path/to/nso-6.4 - $ . ncsrc - $ cd path/to/nso-6.4.1/examples.ncs/service-management/datacenter-qinq - $ ncs -``` - -Since the Runtime Directory is self-contained, this is also the way to move between examples. And since the Runtime Directory is self-contained including the database files, you can compress a complete directory and distribute it. Unpacking that directory and starting NSO from there gives an exact copy of all instance data. - -```bash - $ cd path/to/nso-6.4.1/examples.ncs/service-management/datacenter-qinq - $ ncs - $ ncs --stop - $ cd path/to/nso-6.4.1/examples.ncs/device-management/simulated-cisco-ios - $ ncs - $ ncs --stop -``` - -
- -{% hint style="warning" %} -The `ncs-setup` command creates an `ncs.conf` file that uses predefined encryption keys for easier migration of data across installations. It is not suitable for cases where data confidentiality is required, such as a production deployment. See [Cryptographic Keys](../advanced-topics/cryptographic-keys.md) for ways to generate suitable keys. -{% endhint %} - -### Step 7 - Generate License Registration Token - -To conclude the NSO installation, a license registration token must be created using a (CSSM) account. This is because NSO uses Cisco Smart Licensing, as described in the [Cisco Smart Licensing](../management/system-management/cisco-smart-licensing.md) to make it easy to deploy and manage NSO license entitlements. Login credentials to the [Cisco Smart Software Manager](https://www.cisco.com/c/en/us/buy/smart-accounts/software-manager.html) (CSSM) account are provided by your Cisco contact and detailed instructions on how to [create a registration token](../management/system-management/cisco-smart-licensing.md#d5e2927) can be found in the Cisco Smart Licensing. General licensing information covering licensing models, how licensing works, usage compliance, etc., is covered in the [Cisco Software Licensing Guide](https://www.cisco.com/c/en/us/buy/licensing/licensing-guide.html). - -To generate a license registration token: - -1. When you have a token, start a Cisco CLI towards NSO and enter the token, for example: - - ```bash - $ ncs_cli -Cu admin - admin@ncs# license smart register idtoken YzIzMDM3MTgtZTRkNC00YjkxLTk2ODQt - OGEzMTM3OTg5MG - Registration process in progress. - Use the 'show license status' command to check the progress and result. - ``` - - \ - Upon successful registration, NSO automatically requests a license entitlement for its own instance and for the number of devices it orchestrates and their NED types. If development mode has been enabled, only development entitlement for the NSO instance itself is requested. -2. Inspect the requested entitlements using the command `show license all` (or by inspecting the NSO daemon log). An example output is shown below. - - ```bash - admin@ncs# show license all - ... - 21-Apr-2016::11:29:18.022 miosaterm confd[8226]: - Smart Licensing Global Notification: - type = "notifyRegisterSuccess", - agentID = "sa1", - enforceMode = "notApplicable", - allowRestricted = false, - failReasonCode = "success", - failMessage = "Successful." - 21-Apr-2016::11:29:23.029 miosaterm confd[8226]: - Smart Licensing Entitlement Notification: type = "notifyEnforcementMode", - agentID = "sa1", - notificationTime = "Apr 21 11:29:20 2016", - version = "1.0", - displayName = "regid.2015-10.com.cisco.NSO-network-element", - requestedDate = "Apr 21 11:26:19 2016", - tag = "regid.2015-10.com.cisco.NSO-network-element", - enforceMode = "inCompliance", - daysLeft = 90, - expiryDate = "Jul 20 11:26:19 2016", - requestedCount = 8 - ... - ``` - -
- -Evaluation Period - -If no registration token is provided, NSO enters a 90-day evaluation period, and the remaining evaluation time is recorded hourly in the NSO daemon log: - -``` -... - 13-Apr-2016::13:22:29.178 miosaterm confd[16260]: - Starting the NCS Smart Licensing Java VM - 13-Apr-2016::13:22:34.737 miosaterm confd[16260]: -Smart Licensing evaluation time remaining: 90d 0h 0m 0s -... - 13-Apr-2016::13:22:34.737 miosaterm confd[16260]: - Smart Licensing evaluation time remaining: 89d 23h 0m 0s -... -``` - -
- -
- -Communication Send Error - -During upgrades, if you experience the 'Communication Send Error' with license registration, restart the Smart Agent. - -
- -
- -If You are Unable to Access Cisco Smart Software Manager - -In a situation where the NSO instance has no direct access to the Cisco Smart Software Manager, one option is the [Cisco Smart Software Manager Satellite](https://software.cisco.com/software/csws/ws/platform/home) which can be installed to manage software licenses on the premises. Install the satellite and use the command `call-home destination address http ` to point to the satellite. - -Another option when direct access is not desired is to configure an HTTP or HTTPS proxy, e.g., `smart-license smart-agent proxy url https://127.0.0.1:8080`. If you plan to do this, take the note below regarding ignored CLI configurations into account: - -If `ncs.conf` contains a configuration for any of the java-executable, java-options, override-url/url, or proxy/url under the configure path `/ncs-config/smart-license/smart-agent/`, then any corresponding configuration done via the CLI is ignored. - -
- -
- -License Registration in High Availability (HA) Mode - -When configuring NSO in HA mode, the license registration token must be provided to the CLI running on the primary node. Read more about HA and node types in NSO [High Availability](../management/high-availability.md). - -
- -
- -Licensing Log - -Licensing activities are also logged in the NSO daemon log as described in [Monitoring NSO](../management/system-management/#d5e7876). For example, a successful token registration results in the following log entry: - -``` - 21-Apr-2016::11:29:18.022 miosaterm confd[8226]: - Smart Licensing Global Notification: - type = "notifyRegisterSuccess" -``` - -
- -
- -Check Registration Status - -To check the registration status, use the command `show license status` An example output is shown below. - -```bash -admin@ncs# show license status -Smart Licensing is ENABLED - -Registration: -Status: REGISTERED -Smart Account: Network Services Orchestrator -Virtual Account: Default -Export-Controlled Functionality: Allowed -Initial Registration: SUCCEEDED on Apr 21 09:29:11 2016 UTC -Last Renewal Attempt: SUCCEEDED on Apr 21 09:29:16 2016 UTC -Next Renewal Attempt: Oct 18 09:29:16 2016 UTC -Registration Expires: Apr 21 09:26:13 2017 UTC -Export-Controlled Functionality: Allowed - -License Authorization: -License Authorization: -Status: IN COMPLIANCE on Apr 21 09:29:18 2016 UTC -Last Communication Attempt: SUCCEEDED on Apr 21 09:26:30 2016 UTC -Next Communication Attempt: Apr 21 21:29:32 2016 UTC -Communication Deadline: Apr 21 09:26:13 2017 UTC -``` - -
- -## Local Install FAQs - -Frequently Asked Questions (FAQs) about Local Install. - -
- -Is there a dependency between the NSO Installation Directory and Runtime Directory? - -No, there is no such dependency. - -
- -
- -Do you need to source the ncsrc file before starting NSO? - -Yes. - -
- -
- -Can you start NSO from a directory that is not an NSO runtime directory? - -No. To start NSO, you need to point to the run directory. - -
- -
- -Can you stop NSO from a directory that is not an NSO runtime directory? - -Yes. - -
- -
- -Can we move NSO Installation from one folder to another? - -Yes. You can move the directory where you installed NSO to a new location in your directory tree. Simply move NSO's root directory to the new desired location and update this file: `$NCS_DIR/ncsrc` (and `ncsrc.tcsh` if you want). This is a small and handy script that sets up some environment variables for you. Update the paths to the new location. The `$NCS_DIR/bin/ncs` and `$NCS_DIR/bin/ncsc` scripts will determine the location of NSO's root directory automatically. - -
- -*** - -**Next Steps** - -{% content-ref url="post-install-actions/explore-the-installation.md" %} -[explore-the-installation.md](post-install-actions/explore-the-installation.md) -{% endcontent-ref %} diff --git a/administration/installation-and-deployment/post-install-actions/README.md b/administration/installation-and-deployment/post-install-actions/README.md deleted file mode 100644 index aa3a4d4e..00000000 --- a/administration/installation-and-deployment/post-install-actions/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: Perform actions and activities possible after installing NSO. ---- - -# Post-Install Actions - -The following actions are possible after installing NSO. - -## After Local Install - -{% content-ref url="explore-the-installation.md" %} -[explore-the-installation.md](explore-the-installation.md) -{% endcontent-ref %} - -{% content-ref url="start-stop-nso.md" %} -[start-stop-nso.md](start-stop-nso.md) -{% endcontent-ref %} - -{% content-ref url="create-nso-instance.md" %} -[create-nso-instance.md](create-nso-instance.md) -{% endcontent-ref %} - -{% content-ref url="enable-development-mode.md" %} -[enable-development-mode.md](enable-development-mode.md) -{% endcontent-ref %} - -{% content-ref url="running-nso-examples.md" %} -[running-nso-examples.md](running-nso-examples.md) -{% endcontent-ref %} - -{% content-ref url="migrate-to-system-install.md" %} -[migrate-to-system-install.md](migrate-to-system-install.md) -{% endcontent-ref %} - -{% content-ref url="uninstall-local-install.md" %} -[uninstall-local-install.md](uninstall-local-install.md) -{% endcontent-ref %} - -## After System Install - -{% content-ref url="modify-examples-for-system-install.md" %} -[modify-examples-for-system-install.md](modify-examples-for-system-install.md) -{% endcontent-ref %} - -{% content-ref url="uninstall-system-install.md" %} -[uninstall-system-install.md](uninstall-system-install.md) -{% endcontent-ref %} diff --git a/administration/installation-and-deployment/post-install-actions/create-nso-instance.md b/administration/installation-and-deployment/post-install-actions/create-nso-instance.md deleted file mode 100644 index 8c779b3d..00000000 --- a/administration/installation-and-deployment/post-install-actions/create-nso-instance.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: Create a new NSO instance for Local Install. ---- - -# Create NSO Instance - -{% hint style="warning" %} -Applies to Local Install. -{% endhint %} - -One of the included scripts with an NSO installation is the `ncs-setup`, which makes it very easy to create instances of NSO from a Local Install. You can look at the `--help` or [ncs-setup(1)](../../../resources/man/ncs-setup.1.md) in Manual Pages for more details, but the two options we need to know are: - -* `--dest` defines the directory where you want to set up NSO. if the directory does not exist, it will be created. -* `--package` defines the NEDs that you want to have installed. You can specify this option multiple times. - -{% hint style="info" %} -NCS is the original name of the NSO product. Therefore, many of the commands and application features are prefaced with `ncs`. You can think of NCS as another name for NSO. -{% endhint %} - -To create an NSO instance: - -1. Run the command to set up an NSO instance in the current directory with the IOS, NX-OS, IOS-XR and ASA NEDs. You only need one NED per platform that you want NSO to manage, even if you may have multiple versions in your installer `neds` directory. - - \ - Use the name of the NED folder in `${NCS_DIR}/packages/neds` for the latest NED version that you have installed for the target platform. Use the tab key to complete the path after you start typing (alternatively, copy and paste). Verify that the NED versions below match what is currently on the sandbox to avoid a syntax error. See the example below. - - ```bash - ncs-setup --package ~/nso-6.0/packages/neds/cisco-ios-cli-6.44 \ - --package ~/nso-6.0/packages/neds/cisco-nx-cli-5.15 \ - --package ~/nso-6.0/packages/neds/cisco-iosxr-cli-7.20 \ - --package ~/nso-6.0/packages/neds/cisco-asa-cli-6.8 \ - --dest nso-instance - ``` -2. Check the `nso-instance` directory. Notice that several new files and folders are created. - - ```bash - $ ls nso-instance/ - logs ncs-cdb ncs.conf packages README.ncs scripts state - $ ls -l nso-instance/packages/ - total 0 - lrwxrwxrwx 1 user docker 51 Mar 19 12:44 cisco-asa-cli-6.8 -> - /home/user/nso-6.0/packages/neds/cisco-asa-cli-6.8 - - lrwxrwxrwx 1 user docker 52 Mar 19 12:44 cisco-ios-cli-6.44 -> - /home/user/nso-6.0/packages/neds/cisco-ios-cli-6.44 - - lrwxrwxrwx 1 user docker 54 Mar 19 12:44 cisco-iosxr-cli-7.20 -> - /home/user/nso-6.0/packages/neds/cisco-iosxr-cli-7.20 - - lrwxrwxrwx 1 user docker 51 Mar 19 12:44 cisco-nx-cli-5.15 -> - /home/user/nso-6.0/packages/neds/cisco-nx-cli-5.15 - $ - ``` - - Following is a description of the important files and folders: - - * `ncs.conf` is the NSO application configuration file and is used to customize aspects of the NSO instance (for example, to change ports, enable/disable features, and so on.) See [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) in Manual Pages for information. - * `packages/` is the directory that has symlinks to the NEDs that we referenced in the `--package` arguments at the time of setup. See [NSO Packages](../../../development/core-concepts/packages.md) in Development for more information. - * `logs/` is the directory that contains all the logs from NSO. This directory is useful for troubleshooting. -3. Start the NSO instance by navigating to the `nso-instance` directory and typing the `ncs` command. You must be situated in the `nso-instance` directory each time you want to start or stop NSO. If you have multiple instances, you need to navigate to each one and use the `ncs` command to start or stop each one. -4. Verify that NSO is running by using the `ncs --status | grep status` command. - - ```bash - $ ncs --status | grep status - status: started - db=running id=31 priority=1 path=/ncs:devices/device/live-status-protocol/device-type - ``` -5. Add Netsim or lab devices using the command `ncs-netsim -h`. diff --git a/administration/installation-and-deployment/post-install-actions/enable-development-mode.md b/administration/installation-and-deployment/post-install-actions/enable-development-mode.md deleted file mode 100644 index d4909f59..00000000 --- a/administration/installation-and-deployment/post-install-actions/enable-development-mode.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -description: Enable your NSO instance for development purposes. ---- - -# Enable Development Mode - -{% hint style="warning" %} -Applies to Local Install -{% endhint %} - -If you intend to use your NSO instance for development purposes, enable the development mode using the command `license smart development enable`. diff --git a/administration/installation-and-deployment/post-install-actions/explore-the-installation.md b/administration/installation-and-deployment/post-install-actions/explore-the-installation.md deleted file mode 100644 index 11adb134..00000000 --- a/administration/installation-and-deployment/post-install-actions/explore-the-installation.md +++ /dev/null @@ -1,165 +0,0 @@ ---- -description: Explore NSO contents after finishing the installation. ---- - -# Explore the Installation - -{% hint style="warning" %} -Applies to Local Install. -{% endhint %} - -Before starting NSO, it is recommended to explore the installation contents. - -Navigate to the newly created Installation Directory, for example: - -```bash -cd ~/nso-6.0 -``` - -## Contents of the Installation Directory - -The installation directory includes the following contents: - -* [Documentation](explore-the-installation.md#d5e552) -* [Examples](explore-the-installation.md#d5e560) -* [Network Element Drivers](explore-the-installation.md#d5e564) -* [Shell scripts](explore-the-installation.md#d5e604) - -### Documentation - -Along with the binaries, NSO installs a full set of documentation available in the `doc/` folder in the Installation Directory. There is also an online version of the documentation available from [DevNet](https://developer.cisco.com/docs/nso/nso-fundamentals/). - -```bash -ls -l doc/ -drwxr-xr-x 5 user staff 160B Nov 29 05:19 api/ -drwxr-xr-x 14 user staff 448B Nov 29 05:19 html/ --rw-r--r-- 1 user staff 202B Nov 29 05:19 index.html -drwxr-xr-x 17 user staff 544B Nov 29 05:19 pdf/ -``` - -Run `index.html` in your browser to explore further. - -### Examples - -Local Install comes with a rich set of [examples](https://github.com/NSO-developer/nso-examples/tree/6.6) to start using NSO. - -```bash -$ ls -1 examples.ncs/ -README.md -aaa -common -device-management -getting-started -high-availability -layered-services-architecture -misc -nano-services -northbound-interfaces -scaling-performance -sdk-api -service-management -``` - -### Network Element Drivers (NEDs) - -In order to communicate with the network, NSO uses NEDs as device drivers for different device types. Cisco has NEDs for hundreds of different devices available for customers, and several are included in the installer in the `/packages/neds` directory. - -In the example below, NEDs for Cisco ASA, IOS, IOS XR, and NX-OS are shown. Also included are NEDs for other vendors including Juniper JunOS, A10, ALU, and Dell. - -```bash -$ ls -1 packages/neds -a10-acos-cli-3.0 -alu-sr-cli-3.4 -cisco-asa-cli-6.6 -cisco-ios-cli-3.0 -cisco-ios-cli-3.8 -cisco-iosxr-cli-3.0 -cisco-iosxr-cli-3.5 -cisco-nx-cli-3.0 -dell-ftos-cli-3.0 -juniper-junos-nc-3.0 -``` - -{% hint style="info" %} -The example NEDs included in the installer are intended for evaluation, demonstration, and use with the [examples.ncs](https://github.com/NSO-developer/nso-examples/tree/6.6) examples. These are not the latest versions available and often do not have all the features available in production NEDs. -{% endhint %} - -#### **Install New NEDs** - -A large number of pre-built supported NEDs are available which can be acquired and downloaded by the customers from [Cisco Software Download](https://software.cisco.com/). Note that the specific file names and versions that you download may be different from the ones in this guide. Therefore, remember to update the paths accordingly. - -Like the NSO installer, the NEDs are `signed.bin` files that need to be run to validate the download and extract the new code. - -To install new NEDs: - -1. Change to the working directory where your downloads are. The filenames indicate which version of NSO the NEDs are pre-compiled for (in this case NSO 6.0), and the version of the NED. An example output is shown below. - - ```bash - cd ~/Downloads/ - ls -l ncs*.bin - - # Output - -rw-r--r--@ 1 user staff 9708091 Dec 18 12:05 ncs-6.0-cisco-asa-6.7.7.signed.bin - -rw-r--r--@ 1 user staff 51233042 Dec 18 12:06 ncs-6.0-cisco-ios-6.42.1.signed.bin - -rw-r--r--@ 1 user staff 8400190 Dec 18 12:05 ncs-6.0-cisco-nx-5.13.1.1.signed.bin - ``` -2. Use the `sh` command to run `signed.bin` to verify the certificate and extract the NED tar.gz and other files. Repeat for all files. An example output is shown below. - - ```bash - sh ncs-6.0-cisco-nx-5.13.1.1.signed.bin - - Unpacking... - Verifying signature... - Downloading CA certificate from http://www.cisco.com/security/pki/certs/crcam2.cer ... - Successfully downloaded and verified crcam2.cer. - Downloading SubCA certificate from http://www.cisco.com/security/pki/certs/innerspace.cer ... - Successfully downloaded and verified innerspace.cer. - Successfully verified root, subca and end-entity certificate chain. - Successfully fetched a public key from tailf.cer. - Successfully verified the signature of ncs-6.0-cisco-nx-5.13.1.1.tar.gz using tailf.cer - ``` -3. You now have three tar (.`tar.gz`) files. These are compressed versions of the NEDs. List the files to verify as shown in the example below. - - ```bash - ls -l ncs*.tar.gz - -rw-r--r-- 1 user staff 9704896 Dec 12 21:11 ncs-6.0-cisco-asa-6.7.7.tar.gz - -rw-r--r-- 1 user staff 51260488 Dec 13 22:58 ncs-6.0-cisco-ios-6.42.1.tar.gz - -rw-r--r-- 1 user staff 8409288 Dec 18 09:09 ncs-6.0-cisco-nx-5.13.1.1.tar.gz - ``` -4. Navigate to the `packages/neds` directory for your Local Install, for example: - - ```bash - cd ~/nso-6.0/packages/neds - ``` -5. In the `/packages/neds` directory, extract the .tar files into this directory using the `tar` command with the path to where the compressed NED is located. An example is shown below. - - ``` - tar -zxvf ~/Downloads/ncs-6.0-cisco-nx-5.13.1.1.tar.gz - tar -zxvf ~/Downloads/ncs-6.0-cisco-ios-6.42.1.tar.gz - tar -zxvf ~/Downloads/ncs-6.0-cisco-asa-6.7.7.tar.gz - ``` - - \ - Here is a sample list of the newer NEDs extracted along with the ones bundled with the installation: - - ``` - drwxr-xr-x 13 user staff 416 Nov 29 05:17 a10-acos-cli-3.0 - drwxr-xr-x 12 user staff 384 Nov 29 05:17 alu-sr-cli-3.4 - drwxr-xr-x 13 user staff 416 Nov 29 05:17 cisco-asa-cli-6.6 - drwxr-xr-x 13 user staff 416 Dec 12 21:11 cisco-asa-cli-6.7 - drwxr-xr-x 12 user staff 384 Nov 29 05:17 cisco-ios-cli-3.0 - drwxr-xr-x 12 user staff 384 Nov 29 05:17 cisco-ios-cli-3.8 - drwxr-xr-x 13 user staff 416 Dec 13 22:58 cisco-ios-cli-6.42 - drwxr-xr-x 13 user staff 416 Nov 29 05:17 cisco-iosxr-cli-3.0 - drwxr-xr-x 13 user staff 416 Nov 29 05:17 cisco-iosxr-cli-3.5 - drwxr-xr-x 13 user staff 416 Nov 29 05:17 cisco-nx-cli-3.0 - drwxr-xr-x 14 user staff 448 Dec 18 09:09 cisco-nx-cli-5.13 - drwxr-xr-x 13 user staff 416 Nov 29 05:17 dell-ftos-cli-3.0 - drwxr-xr-x 10 user staff 320 Nov 29 05:17 juniper-junos-nc-3.0 - ``` - -### Shell Scripts - -The last thing to note is the files `ncsrc` and `ncsrc.tsch`. These are shell scripts for `bash` and `tsch` that set up your PATH and other environment variables for NSO. Depending on your shell, you need to source this file before starting NSO. - -For more information on sourcing shell script, see the [Local Install steps](../local-install.md). diff --git a/administration/installation-and-deployment/post-install-actions/migrate-to-system-install.md b/administration/installation-and-deployment/post-install-actions/migrate-to-system-install.md deleted file mode 100644 index 55522933..00000000 --- a/administration/installation-and-deployment/post-install-actions/migrate-to-system-install.md +++ /dev/null @@ -1,125 +0,0 @@ ---- -description: Convert your current Local Install setup to a System Install. ---- - -# Migrate to System Install - -{% hint style="warning" %} -Applies to Local Install. -{% endhint %} - -If you already have a Local Install with existing data that you would like to convert into a System Install, the following procedure allows you to do so. However, a reverse migration from System to Local Install is not supported. - -{% hint style="info" %} -It is possible to perform the migration and upgrade simultaneously to a newer NSO version, however, doing so introduces additional complexity. If you run into issues, first migrate, and then perform the upgrade. -{% endhint %} - -The following procedure assumes that NSO is installed as described in the NSO Local Install process and will perform an initial System Install of the same NSO version. After following these steps, consult the NSO System Install guide for additional steps that are required for a fully functional System Install. - -The procedure also assumes you are using the `$HOME/ncs-run` folder as the run directory. If this is not the case, modify the following path accordingly. - -To migrate to System Install: - -1. Stop the current (local) NSO instance if it is running. - - ```bash - $ ncs --stop - ``` -2. Take a complete backup of the Runtime Directory for potential disaster recovery. - - ```bash - $ tar -czf $HOME/ncs-backup.tar.gz -C $HOME ncs-run - ``` -3. Change to Super User privileges. - - ```bash - $ sudo -s - ``` -4. Start the NSO System Install. - - ```bash - $ sh nso-VERSION.OS.ARCH.installer.bin --system-install - ``` -5. If you have multiple versions of NSO installed, verify that the symbolic link in `/opt/ncs` points to the correct version. -6. Copy the CDB files containing data to the central location. - - ```bash - # cp $HOME/ncs-run/ncs-cdb/*.cdb /var/opt/ncs/cdb - ``` -7. Ensure that the `/var/opt/ncs/packages` directory includes all the necessary packages, appropriate for the NSO version. However, copying the packages directly could later on interfere with the operation of the `nct` command. It is better to only use symbolic links in that folder. Instead, copy the existing packages to the `/opt/ncs/packages` directory, either as directories or as tarball files. Make sure that each package includes the NSO version in its name and is not just a symlink, for example: - - ```bash - # cd $HOME/ncs-run/packages - # for pkg in *; do cp -RL $pkg /opt/ncs/packages/ncs-VERSION-$pkg; done - ``` -8. Link to these packages in the `/var/opt/ncs/packages` directory. - - ```bash - # cd /var/opt/ncs/packages/ - # rm -f * - # for pkg in /opt/ncs/packages/ncs-VERSION-*; do ln -s $pkg; done - ``` - - \ - The reason for prepending `ncs-VERSION` to the filename is to allow additional NSO commands, such as `nct upgrade` and `software packages` to work properly. These commands need to identify which NSO version a package was compiled for. -9. Edit the `/etc/ncs/ncs.conf` configuration file and make the necessary changes. If you wish to use the configuration from Local Install, disable the local authentication, unless you fully understand its security implications. - - ```xml - - false - - ``` -10. When starting NSO at boot using `systemd`, make sure that you set the package reload option from the `/etc/ncs/ncs.systemd.conf` environment file to `true`. Or, for example, set `NCS_RELOAD_PACKAGES=true` before starting NSO if using the `ncs` command. - - ```bash - # systemctl daemon-reload - # systemctl start ncs - ``` -11. Review and complete the steps in NSO System Install, except running the installer, which you have done already. Once completed, you should have a running NSO instance with data from the Local Install. -12. Remove the package reload option if it was set. - - ```bash - # unset NCS_RELOAD_PACKAGES - ``` -13. Update log file paths for Java and Python VM through the NSO CLI. - - ```bash - $ ncs_cli -C -u admin - admin@ncs# config - Entering configuration mode terminal - admin@ncs(config)# unhide debug - admin@ncs(config)# show full-configuration java-vm stdout-capture file - java-vm stdout-capture file ./logs/ncs-java-vm.log - admin@ncs(config)# java-vm stdout-capture file /var/log/ncs/ncs-java-vm.log - admin@ncs(config)# commit - Commit complete. - admin@ncs(config)# show full-configuration java-vm stdout-capture file - java-vm stdout-capture file /var/log/ncs/ncs-java-vm.log - admin@ncs(config)# show full-configuration python-vm logging log-file-prefix - python-vm logging log-file-prefix ./logs/ncs-python-vm - admin@ncs(config)# python-vm logging log-file-prefix /var/log/ncs/ncs-python-vm - admin@ncs(config)# commit - Commit complete. - admin@ncs(config)# show full-configuration python-vm logging log-file-prefix - python-vm logging log-file-prefix /var/log/ncs/ncs-python-vm - admin@ncs(config)# exit - admin@ncs# - admin@ncs# exit - ``` -14. Verify that everything is working correctly. - -At this point, you should have a complete copy of the previous Local Install running as a System Install. Should the migration fail at some point and you want to back out of it, the Local Install was not changed and you can easily go back to using it as before. - -```bash -$ sudo systemctl stop ncs -$ source $HOME/ncs-VERSION/ncsrc -$ cd $HOME/ncs-run -$ ncs -``` - -In the unlikely event of Local Install becoming corrupted, you can restore it from the backup. - -```bash -$ rm -rf $HOME/ncs-run -$ tar -xzf $HOME/ncs-backup.tar.gz -C $HOME -``` diff --git a/administration/installation-and-deployment/post-install-actions/modify-examples-for-system-install.md b/administration/installation-and-deployment/post-install-actions/modify-examples-for-system-install.md deleted file mode 100644 index 46efe5da..00000000 --- a/administration/installation-and-deployment/post-install-actions/modify-examples-for-system-install.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -description: Alter your examples to work with System Install. ---- - -# Modify Examples for System Install - -{% hint style="warning" %} -Applies to System Install. -{% endhint %} - -Since all the NSO examples and README steps that come with the installer are primarily aimed at Local Install, you need to modify them to run them on a System Install. - -To work with the System Install structure, this may require a little or bigger modification depending on the example. - -For example, to port the [example.ncs/nano-services/basic-vrouter](https://github.com/NSO-developer/nso-examples/tree/6.6/nano-services/basic-vrouter) example to the System Install structure: - -1. Make the following changes to the `basic-vrouter/ncs.conf` file: - - ```xml - false - 0.0.0.0 - 8888 - -${NCS_DIR}/etc/ncs/ssl/cert/host.key - -${NCS_DIR}/etc/ncs/ssl/cert/host.cert - +${NCS_CONFIG_DIR}/etc/ncs/ssl/cert/host.key - +${NCS_CONFIG_DIR}/etc/ncs/ssl/cert/host.cert -
- - ``` -2. Copy the Local Install `$NCS_DIR/var/ncs/cdb/aaa_init.xml` file to the `basic-vrouter/` folder. - -Other, more complex examples may require more `ncs.conf` file changes or require a copy of the Local Install default `$NCS_DIR/etc/ncs/ncs.conf` file together with the modification described above, or require the Local Install tool `$NCS_DIR/bin/ncs-setup` to be installed, as the `ncs-setup` command is usually not useful with a System Install. See [Migrate to System Install](migrate-to-system-install.md) for more information. diff --git a/administration/installation-and-deployment/post-install-actions/running-nso-examples.md b/administration/installation-and-deployment/post-install-actions/running-nso-examples.md deleted file mode 100644 index ee16338c..00000000 --- a/administration/installation-and-deployment/post-install-actions/running-nso-examples.md +++ /dev/null @@ -1,145 +0,0 @@ ---- -description: Run and interact with practice examples provided with the NSO installer. ---- - -# Running NSO Examples - -{% hint style="warning" %} -Applies to Local Install. -{% endhint %} - -This section provides an overview of how to run the examples provided with the NSO installer. By working through the examples, the reader should get a good overview of the various aspects of NSO and hands-on experience from interacting with it. - -{% hint style="info" %} -This section references the examples located in [$NCS\_DIR/examples.ncs](https://github.com/NSO-developer/nso-examples/tree/6.6). The examples all have `README` files that include instructions related to the example. -{% endhint %} - -## General Instructions - -1. Make sure that NSO is installed with a Local Install according to the instructions in [Local Install](../local-install.md). -2. Source the `ncsrc` file in the NSO installation directory to set up a local environment. For example: - - ```bash - $ source ~/nso-6.0/ncsrc - ``` -3. Proceed to the example directory: - - ```bash - $ cd $NCS_DIR/examples.ncs/device-management/simulated-cisco-ios - ``` -4. Follow the instructions in the `README` files that are located in the example directories. - -Every example directory is a complete NSO run-time directory. The README file and the detailed instructions later in this guide show how to generate a simulated network and NSO configuration for running the specific examples. Basically, the following steps are done: - -1. Create a simulated network using the `ncs-netsim --create-network` command: - - ```bash - $ ncs-netsim create-network cisco-ios-cli-3.8 3 ios - ``` - - This creates 3 Cisco IOS devices called `ios0`, `ios1`, and `ios2`. -2. Create an NSO run-time environment using the `ncs-setup` command: - - ```bash - $ ncs-setup --dest . - ``` - - This command uses the `--dest` option to create local directories for logs, database files, and the NSO configuration file to the current directory (note that `.` refers to the current directory). -3. Start NCS netsim: - - ```bash - $ ncs-netsim start - ``` -4. Start NSO: - - ```bash - $ ncs - ``` - -{% hint style="warning" %} -It is important to make sure that you stop `ncs` and `ncs-netsim` when moving between examples using the `stop` option of the `netsim` and the `--stop` option of the `ncs`. - -```bash -$ cd $NCS_DIR/examples.ncs/device-management/simulated-cisco-ios -$ ncs-netsim start -$ ncs -$ ncs-netsim stop -$ ncs --stop -``` -{% endhint %} - -## Common Mistakes - -Some of the most common mistakes are: - -
- -Not Sourcing the ncsrc File - -You have not sourced the `ncsrc` file: - -```bash -$ ncs --bash: ncs: command not found -``` - -
- -
- -Not Starting NSO from the Runtime Directory - -You are trying to start NSO from a directory that is not set up as a runtime directory. - -```bash -$ ncs -Bad configuration: /etc/ncs/ncs.conf:0: "./state/packages-in-use: \ - Failed to create symlink: no such file or directory" -Daemon died status=21 -``` - -What happened above is that NSO did not find an `ncs.conf` in the local directory, so it uses the default in `/etc/ncs/ncs.conf`. That `ncs.conf` says there shall be directories at `./` such as `./state` which is not true. Make sure that you `cd` to the root of the example and check that there is a `ncs.conf` file and a `cdb-dir` directory. - -
- -
- -Having Another Instance of NSO Running - -You already have another instance of NSO running (or the same with netsim): - -```bash -$ ncs -Cannot bind to internal socket 127.0.0.1:4569 : address already in use -Daemon died status=20 -$ ncs-netsim start -DEVICE c0 Cannot bind to internal socket 127.0.0.1:5010 : \ - address already in use -Daemon died status=20 -FAIL -``` - -To resolve the above, just stop the running instance of NSO and netsim. Remember that it does not matter where you started the "running" NSO and netsim; there is no need to `cd` back to the other example before stopping. - -
- -
- -Not Having the NetSim Device Configuration Loaded into NSO - -Another problem that users run into sometimes is where the NetSim device configuration is not loaded into NSO. This can happen if the order of commands is not followed. To resolve this, remove the database files in the `ncs_cdb` directory (keep any files with the `.xml` extension). In this way, NSO will reload the XML initialization files provided by **ncs-setup**. - -```bash -$ ncs --stop -$ cd ncs-cdb/ -$ ls -A.cdb -C.cdb -O.cdb -S.cdb -netsim_devices_init.xml -$ rm *.cdb -$ ncs -``` - -
diff --git a/administration/installation-and-deployment/post-install-actions/start-stop-nso.md b/administration/installation-and-deployment/post-install-actions/start-stop-nso.md deleted file mode 100644 index 93030e72..00000000 --- a/administration/installation-and-deployment/post-install-actions/start-stop-nso.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -description: Start and stop the NSO daemon. ---- - -# Start and Stop NSO - -{% hint style="warning" %} -Applies to Local Install. -{% endhint %} - -The command `ncs -h` shows various options when starting NSO. By default, NSO starts in the background without an associated terminal. It is recommended to add NSO to the `/etc/init` scripts of the deployment hosts. For more information, see the [ncs(1)](../../../resources/man/ncs.1.md) in Manual Pages. - -Whenever you start (or reload) the NSO daemon, it reads its configuration from `./ncs.conf` or `${NCS_DIR}/etc/ncs/ncs.conf` or from the file specified with the `-c` option. Parts of the configuration can also be placed in the `ncs.conf.d` directory that must be placed next to the actual `ncs.conf` file. - -```bash -$ ncs -$ ncs --stop -$ ncs -h -... -``` diff --git a/administration/installation-and-deployment/post-install-actions/uninstall-local-install.md b/administration/installation-and-deployment/post-install-actions/uninstall-local-install.md deleted file mode 100644 index d0287273..00000000 --- a/administration/installation-and-deployment/post-install-actions/uninstall-local-install.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -description: Remove Local Install. ---- - -# Uninstall Local Install - -{% hint style="warning" %} -Applies to Local Install. -{% endhint %} - -To uninstall Local Install, simply delete the Install Directory. diff --git a/administration/installation-and-deployment/post-install-actions/uninstall-system-install.md b/administration/installation-and-deployment/post-install-actions/uninstall-system-install.md deleted file mode 100644 index f5a319f4..00000000 --- a/administration/installation-and-deployment/post-install-actions/uninstall-system-install.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -description: Remove System Install. ---- - -# Uninstall System Install - -{% hint style="warning" %} -Applies to System Install. -{% endhint %} - -NSO can be uninstalled using the [ncs-installer(1)](../../../resources/man/ncs-installer.1.md) option only if NSO is installed with `--system-install` option. Either part of the static files or the full installation can be removed using `ncs-uninstall` option. Ensure to stop NSO before uninstalling. - -```bash -# ncs-uninstall --all -``` - -Executing the above command removes the Installation Directory `/opt/ncs` including symbolic links, Configuration Directory `/etc/ncs`, Run Directory `/var/opt/ncs`, Log Directory `/var/log/ncs`, `systemd` service file `/etc/systemd/system/ncs.service`, `systemd`environment file `/etc/ncs/ncs.systemd.conf`, and the user profile scripts from `/etc/profile.d`. - -To make sure that no license entitlements are consumed after you have uninstalled NSO, be sure to perform the `deregister` command in the CLI: - -```cli -admin@ncs# license smart deregister -``` diff --git a/administration/installation-and-deployment/system-install.md b/administration/installation-and-deployment/system-install.md deleted file mode 100644 index f0d5a5ea..00000000 --- a/administration/installation-and-deployment/system-install.md +++ /dev/null @@ -1,816 +0,0 @@ ---- -description: Install NSO for production use in a system-wide deployment. ---- - -# System Install - -## Installation Steps - -Complete the following activities in the given order to perform a System Install of NSO. - -
Prepare1. Fulfill System Requirements
2. Download Installer/NEDs
3. Unpack the Installer
Install4. Run the Installer
Finalize5. Set up User Access
6. Set Environment Variables
7. Runtime Directory Creation
8. Generate License Token
- -{% hint style="info" %} -**Mode of Install** - -NSO System Install can be installed in **standard mode** or in [**FIPS**](https://www.nist.gov/itl/publications-0/federal-information-processing-standards-fips)**-compliant mode**. Standard mode install supports a broader set of cryptographic algorithms, while the FIPS mode install restricts NSO to use only FIPS 140-3-validated cryptographic modules and algorithms for enhanced/regulated security and compliance. Use FIPS mode only in environments that require compliance with specific security standards, especially in U.S. federal agencies or regulated industries. For all other use cases, install NSO in standard mode. - -\* FIPS: Federal Information Processing Standards -{% endhint %} - -### Step 1 - Fulfill System Requirements - -Start by setting up your system to install and run NSO. - -To install NSO: - -1. Fulfill at least the primary requirements. -2. If you intend to build and run NSO deployment examples, you also need to install additional applications listed under Additional Requirements. - -{% hint style="warning" %} -Where requirements list a specific or higher version, there always exists a (small) possibility that a higher version introduces breaking changes. If in doubt whether the higher version is fully backwards compatible, always use the specific version. -{% endhint %} - -
- -Primary Requirements - -Primary requirements to do a System Install include: - -* A system running Linux or macOS on either the `x86_64` or `ARM64` architecture for development. Linux for production. For [FIPS](https://www.nist.gov/itl/publications-0/federal-information-processing-standards-fips) mode, OS FIPS compliance may be required depending on your specific requirements. -* GNU libc 2.24 or higher. -* Java JRE 17 or higher. Used by Cisco Smart Licensing. -* Required and included with many Linux/macOS distributions: - * `tar` command. Unpack the installer. - * `gzip` command. Unpack the installer. - * `ssh-keygen` command. Generate SSH host key. - * `openssl` command. Generate self-signed certificates for HTTPS. - * `find` command. Used to find out if all required libraries are available. - * `which` command. Used by the NSO package manager. - * `libpam.so.0`. Pluggable Authentication Module library. - * `libexpat.so.1`. EXtensible Markup Language parsing library. - * `libz.so.1` version 1.2.7.1 or higher. Data compression library. - -
- -
- -Additional Requirements - -Additional requirements to, for example, build and run NSO production deployment examples include: - -* Java JDK 17 or higher. -* Ant 1.9.8 or higher. -* Python 3.10 or higher. -* Python Setuptools is required to build the Python API. -* Often installed using the Python package installer pip: - * Python Paramiko 2.2 or higher. To use netconf-console. - * Python requests. Used by the RESTCONF demo scripts. -* `xsltproc` command. Used by the `support/ned-make-package-meta-data` command to generate the `package-meta-data.xml` file. -* One of the following web browsers is required for NSO GUI capabilities. The version must be supported by the vendor at the time of release. - * Safari - * Mozilla Firefox - * Microsoft Edge - * Google Chrome -* OpenSSH client applications. For example, `ssh` and `scp` commands. -* cron. Run time-based tasks, such as `logrotate`. -* `logrotate`. rotate, compress, and mail NSO and system logs. -* `rsyslog`. pass NSO logs to a local syslog managed by `rsyslogd` and pass logs to a remote node. -* `systemd` or `init.d` scripts to start and stop NSO. - -
- -
- -FIPS Mode Entropy Requirements - -The following applies if you are running a container-based setup of your FIPS install: - -In containerized environments (e.g., Docker) that run on older Linux kernels (e.g., Ubuntu 18.04), `/dev/random` may block if the system’s entropy pool is low. This can lead to delays or hangs in FIPS mode, as cryptographic operations require high-quality randomness. - -To avoid this: - -* Prefer newer kernels (e.g., Ubuntu 22.04 or later), where entropy handling is improved to mitigate the issue. -* Or, install an entropy daemon like Haveged on the Docker host to help maintain sufficient entropy. - -Check available entropy on the host system with: - -```bash -cat /proc/sys/kernel/random/entropy_avail -``` - -A value of 256 or higher is generally considered safe. Reference: [Oracle blog post](https://blogs.oracle.com/linux/post/entropyavail-256-is-good-enough-for-everyone). - -
- -### Step 2 - Download the Installer and NEDs - -To download the Cisco NSO installer and example NEDs: - -1. Go to the Cisco's official [Software Download](https://software.cisco.com/download/home) site. -2. Search for the product "Network Services Orchestrator" and select the desired version. -3. There are two versions of the NSO installer, i.e. for macOS and Linux systems. For System Install, choose the Linux OS version. - -
- -Identifying the Installer - -You need to know your system specifications (Operating System and CPU architecture) to choose the appropriate NSO installer. - -NSO is delivered as an OS/CPU-specific signed self-extractable archive. The signed archive file has the pattern `nso-VERSION.OS.ARCH.signed.bin` that after signature verification extracts the `nso-VERSION.OS.ARCH.installer.bin` archive file, where: - -* `VERSION` is the NSO version to install. -* `OS` is the Operating System (`linux` for all Linux distributions and `darwin` for macOS). -* `ARCH` is the CPU architecture, for example`x86_64`. - -
- -### Step 3 - Unpack the Installer - -If your downloaded file is a `signed.bin` file, it means that it has been digitally signed by Cisco, and upon execution, you will verify the signature and unpack the `installer.bin`. - -If you only have `installer.bin`, skip to the next step. - -To unpack the installer: - -1. In the terminal, list the binaries in the directory where you downloaded the installer, for example: - - ```bash - cd ~/Downloads - ls -l nso*.bin - -rw-r--r--@ 1 user staff 199M Dec 15 11:45 nso-6.0.linux.x86_64.installer.bin - -rw-r--r--@ 1 user staff 199M Dec 15 11:45 nso-6.0.linux.x86_64.signed.bin - ``` -2. Use the `sh` command to run the `signed.bin` to verify the certificate and extract the installer binary and other files. An example output is shown below. - - ```bash - sh nso-6.0.linux.x86_64.signed.bin - # Output - Unpacking... - Verifying signature... - Downloading CA certificate from http://www.cisco.com/security/pki/certs/crcam2.cer ... - Successfully downloaded and verified crcam2.cer. - Downloading SubCA certificate from http://www.cisco.com/security/pki/certs/innerspace.cer ... - Successfully downloaded and verified innerspace.cer. - Successfully verified root, subca and end-entity certificate chain. - Successfully fetched a public key from tailf.cer. - Successfully verified the signature of nso-6.0.linux.x86_64.installer.bin using tailf.cer - ``` -3. List the files to check if extraction was successful. - - ```bash - ls -l - # Output - -rw-r--r-- 1 user staff 1.8K Nov 29 06:05 README.signature - -rw-r--r-- 1 user staff 12K Nov 29 06:05 cisco_x509_verify_release.py - -rwxr-xr-x 1 user staff 199M Nov 29 05:55 nso-6.0.linux.x86_64.installer.bin - -rw-r--r-- 1 user staff 256B Nov 29 06:05 nso-6.0.linux.x86_64.installer.bin.signature - -rwxr-xr-x@ 1 user staff 199M Dec 15 11:45 nso-6.0.linux.x86_64.signed.bin - -rw-r--r-- 1 user staff 1.4K Nov 29 06:05 tailf.cer - ``` - -{% hint style="info" %} -There may also be additional files present. -{% endhint %} - -
- -Description of Unpacked Files - -The following contents are unpacked: - -* `nso-VERSION.OS.ARCH.installer.bin`: The NSO installer. -* `nso-VERSION.OS.ARCH.installer.bin.signature`: Signature generated for the NSO image. -* `tailf.cer`: An enclosed Cisco-signed x.509 end-entity certificate containing the public key that is used to verify the signature. -* `README.signature`: File with further details on the unpacked content and steps on how to run the signature verification program. To manually verify the signature, refer to the steps in this file. -* `cisco_x509_verify_release.py`: Python program that can be used to verify the 3-tier x.509 certificate chain and signature. -* Multiple `.tar.gz` files: Bundled packages, extending the base NSO functionality. -* Multiple `.tar.gz.signature` files: Digital signatures for the bundled packages. - -Since NSO version 6.3, a few additional NSO packages are included. They contain the following platform tools: - -* HCC -* Observability Exporter -* Phased Provisioning -* Resource Manager - -For platform tools documentation, refer to the individual package's `README` file or to the [online documentation](https://nso-docs.cisco.com/resources). - -**NED Packages** - -The NED packages that are available with the NSO installation are netsim-based example NEDs. These NEDs are used for NSO examples only. - -Fetch the latest production-grade NEDs from [Cisco Software Download](https://software.cisco.com/download/home) using the URLs provided on your NED license certificates. - -**Manual Pages** - -The installation program will unpack the NSO manual pages from the documentation archive, allowing you to use the `man` command to view them. The Manual Pages are also available in PDF format and from the online documentation located on [NCS man-pages, Volume 1](../../resources/man/ncs-installer.1.md) in Manual Pages. - -Following is a list of a few of the installed manual pages: - -* `ncs(1)`: Command to start and control the NSO daemon. -* `ncsc(1)`: NSO Yang compiler. -* `ncs_cli(1)`: Frontend to the NSO CLI engine. -* `ncs-netsim(1)`: Command to create and manipulate a simulated network. -* `ncs-setup(1)`: Command to create an initial NSO setup. -* `ncs.conf`: NSO daemon configuration file format. - -For example, to view the manual page describing the NSO configuration file, you should type: - -```bash -$ man ncs.conf -``` - -Apart from the manual pages, extensive information about command line options can be obtained by running `ncs` and `ncsc` with the `--help` (abbreviated `-h`) flag. - -```bash -$ ncs --help -``` - -```bash -$ ncsc --help -``` - -**Installer Help** - -Run the `sh nso-VERSION.linux.x86_64.installer.bin --help` command to view additional help on running binaries. More details can be found in the [ncs-installer(1)](../../resources/man/ncs-installer.1.md) Manual Page included with NSO. - -Notice the two options for `--local-install` or `--system-install`. - -```bash -sh nso-6.0.linux.x86_64.installer.bin --help -``` - -
- -### Step 4 - Run the Installer - -To run the installer: - -1. Navigate to your Install Directory. -2. Run the installer with the `--system-install` option to perform System Install. This option creates an install of NSO that is suitable for production deployment. At this point, you can choose to install NSO in standard mode or in FIPS mode. - -{% tabs %} -{% tab title="Standard System Install" %} -The standard mode is the regular NSO install and is suitable for most installations. FIPS is disabled in this mode. - -For standard NSO install, run the installer as below. - -```bash -$ sudo sh nso-VERSION.OS.ARCH.installer.bin --system-install -``` - -{% code title="Example: Standard System Install" %} -```bash -$ sudo sh nso-6.0.linux.x86_64.installer.bin --system-install -``` -{% endcode %} -{% endtab %} - -{% tab title="FIPS System Install" %} -FIPS mode creates a FIPS-compliant NSO install. - -FIPS mode should only be used for deployments that are subject to strict compliance regulations as the cryptographic functions are then confined to the CiscoSSL FIPS 140-3 module library. - -For FIPS-compliant NSO install, run the command with the additional `--fips-install` flag. Afterwards, verify FIPS in `ncs.conf`. - -```bash -$ sudo sh nso-VERSION.OS.ARCH.installer.bin --system-install --fips-install -``` - -{% code title="Example: FIPS System Install" %} -```bash -$ sudo sh nso-6.5.linux.x86_64.installer.bin --system-install --fips-install -``` -{% endcode %} - -{% hint style="info" %} -**NSO Configuration for FIPS** - -Note the following as part of FIPS-specific configuration/install: - -1. The `ncs.conf` file is automatically configured to enable FIPS by setting the following flag: - -```xml - - true - -``` - -2. Additional environment variables (`NCS_OPENSSL_CONF_INCLUDE`, `NCS_OPENSSL_CONF`, `NCS_OPENSSL_MODULES`) are configured in `ncsrc` for FIPS compliance. -3. The default `crypto.so` is overwritten at install for FIPS compliance. - -Additionally, note that: - -* As certain algorithms typically available with CiscoSSL are not included in the FIPS 140-3 validated module (and therefore disabled in FIPS mode), you need to configure NSO to use only the algorithms and cryptographic suites available through the CiscoSSL FIPS 140-3 object module. -* With FIPS, NSO signals the NEDs to operate in FIPS mode using Bouncy Castle FIPS libraries for Java-based components, ensuring compliance with FIPS 140-3. To support this, NED packages may also require upgrading, as older versions — particularly SSH-based NEDs — often lack the necessary FIPS signaling or Bouncy Castle support required for cryptographic compliance. -* Configure SSH keys in `ncs.conf` and `init.xml`. -{% endhint %} -{% endtab %} -{% endtabs %} - -
- -Default Directories and Scripts - -The System Install by default creates the following directories: - -* The Installation Directory is created in `/opt/ncs`, where the distribution is available. -* The Configuration Directory is created in `/etc/ncs`, where the `ncs.conf` file, SSH keys, and WebUI certificates are created. -* The Running Directory is created in `/var/opt/ncs`, where runtime state files, CDB database, and packages are created. -* The Log Directory is created in `/var/log/ncs`, where the log files are populated. -* System-wide environment variables are created in `/etc/profile.d/ncs.sh`. -* The installer creates a `systemd` system service script in `/etc/systemd/system/ncs.service` and enables the NSO service to start at boot, but the service is _not_ started immediately. See the steps below for starting NSO after installation and before rebooting. -* To allow package reload when starting NSO, an environment file called `/etc/ncs/ncs.systemd.conf` is created. This file is owned by the user that starts NSO. - -For the `--system-install` option, you can also choose a user-defined (non-default) Installation Directory, Configuration Directory, Running Directory, and Log Directory with `--install-dir`, `--config-dir`, `--run-dir` and `--log-dir` parameters, and specify that NSO should run as a different user than root with the `--run-as-user` parameter. - -If you choose a non-default Installation Directory by using `--install-dir`, you need to specify `--install-dir` for subsequent installs and also for backup and restore. - -Use the `--ignore-init-scripts` option to disable provisioning the `systemd` system service. - -If a legacy SysV service exists in `/etc/init.d/ncs` when installing in interactive mode, the user will be prompted to continue using the old SysV service behavior or prepare a `systemd` service. In non-interactive mode, a `systemd` service will be prepared where a `/etc/systemd/system/ncs.service.prepare` file is created. The service is not enabled to start at boot. To enable it, rename it to `/etc/systemd/system/ncs.service` and remove the old `/etc/init.d/ncs` SysV service. When using the `--non-interactive` option, the `/etc/systemd/system/ncs.service` file will be overwritten if it already exists. - -For more information on the `ncs-installer`, see the [ncs-installer(1)](../../resources/man/ncs-installer.1.md) man page. - -For an extensive guide to NSO deployment, refer to [Development to Production Deployment](development-to-production-deployment/)_._ - -
- -
- -Enable Strict Overcommit Accounting on the Host. - -By default, the Linux kernel allows overcommit of memory. However, memory overcommit produces an unexpected and unreliable environment for NSO because the Linux Out-Of-Memory (OOM) killer may terminate NSO without restarting it if the system is critically low on memory. Also, when the OOM killer terminates NSO, no system dump file will be produced, and the debug information will be lost. Thus, it is strongly recommended to enable strict overcommit accounting. - -#### **Heuristic Overcommit Mode as an Alternative to Strict Overcommit** - -The alternative—using heuristic overcommit mode (see below for best‑effort recommendations)—can be useful if the NSO host has severe memory limitations. For example, if RAM sizing for the NSO host did not take into account that the schema (from YANG models) is loaded into memory by NSO Python and Java packages affecting total committed memory (Committed\_AS) and after considering the recommendations in [CDB Stores the YANG Model Schema](../../development/advanced-development/scaling-and-performance-optimization.md#d5e8743). - -#### Recommended: Host Configured for Strict Overcommit - -* Set `vm.overcommit_memory=2` to enable strict overcommit accounting. -* Set `vm.overcommit_ratio` so the CommitLimit is approximately equal to physical RAM, with a 5% headroom for the kernel to reduce the risk of system-wide OOM conditions. E.g., 95% of RAM when no swap is present (recommended), or subtract 5 percentage points from the calculated ratio that neutralizes swap. Increase the headroom if the host runs additional services. -* Alternatively, set `vm.overcommit_kbytes` which takes precedence; `vm.overcommit_ratio` is ignored while `vm.overcommit_kbytes > 0`. - * When vm.overcommit\_kbytes > 0, it sets a fixed CommitLimit in kB and ignores ratio and swap in the calculation. Note that HugeTLB is not subtracted when overcommit\_kbytes is used (it’s a fixed value). -* Strongly discourage swap use at runtime by setting `vm.swappiness=1`. -* If swap must remain enabled system-wide, prevent NSO from using swap by configuring its cgroup with `memory.swap.max=0` (cgroup v2). -* If swap must be enabled for NSO use a fast disk, for example, an NVMe SSD. - -**Apply Immediately** - -{% code title="To apply strict overcommit accounting with immediate effect" %} -```bash -echo 2 > /proc/sys/vm/overcommit_memory -``` -{% endcode %} - -When `vm.overcommit_memory=2`, the overcommit\_ratio parameter defines the percentage of physical RAM that is available for commit. - -The Linux kernel computes the CommitLimit: - -CommitLimit = MemTotal × (overcommit\_ratio / 100) + SwapTotal − total\_huge\_TLB - -* MemTotal is the total amount of RAM on the system. -* overcommit\_ratio is the value in `/proc/sys/vm/overcommit_ratio` . -* SwapTotal is the amount of swap space. Can be 0. -* total\_huge\_TLB is the amount of memory set aside for huge pages. Can be 0. - -The default overcommit\_ratio is 50%. On systems with more than 50% of RAM available, this default can underutilize physical memory. - -Do not set `vm.overcommit_ratio=100` as it includes all RAM plus all swap in the CommitLimit and leaves no headroom for the kernel. While swap increases the commit capacity, it is usually slow and should be avoided for NSO. - -**Compute overcommit\_ratio to Neutralize Swap** - -To allocate physical RAM only in commit accounting and keep a 5-10% headroom for the kernel: - -* Compute the base ratio: base\_ratio = 100 × (MemTotal − SwapTotal) / MemTotal. -* Apply headroom: overcommit\_ratio = floor(base\_ratio) − 5. - -Notes: - -* overcommit\_ratio is an integer; round down for a bit of extra headroom. -* Recompute the ratio if RAM or swap changes. -* If SwapTotal ≥ MemTotal, swap cannot be neutralized via overcommit\_ratio, use overcommit\_kbytes; see Example 3. -* If the computed value is very low, ensure it still fits your workload requirements. - -**Example 1: No Swap, 5% Headroom** - -{% code title="Check memory totals" %} -```bash -cat /proc/meminfo | grep "MemTotal\|SwapTotal" -MemTotal: 8039352 kB -SwapTotal: 0 kB -``` -{% endcode %} - -{% code title="Apply settings with immediate effect" %} -```bash -echo 2 > /proc/sys/vm/overcommit_memory -echo 95 > /proc/sys/vm/overcommit_ratio -echo 1 > /proc/sys/vm/swappiness -``` -{% endcode %} - -Rationale: With no swap, set overcommit\_ratio=95 to allow \~95% of RAM for user-space commit, leaving \~5% headroom for the kernel. - -**Example 2: MemTotal > SwapTotal, Neutralize Swap with 5% Headroom** - -{% code title="Check memory totals" %} -```bash -cat /proc/meminfo | grep "MemTotal\|SwapTotal" -MemTotal: 8039352 kB -SwapTotal: 1048572 kB -``` -{% endcode %} - -Calculate the ratio: - -* base\_ratio= 100 × ((8039352 − 1048572) / 8039352) ≈ 86.9%. -* Apply 5% headroom: overcommit\_ratio = floor(86.9) − 5 = 81. - -{% code title="Apply" %} -```bash -echo 2 > /proc/sys/vm/overcommit_memory -echo 81 > /proc/sys/vm/overcommit_ratio -echo 1 > /proc/sys/vm/swappiness -``` -{% endcode %} - -This keeps the CommitLimit safely below physical RAM to provide kernel headroom and neutralizes swap’s contribution to CommitLimit and then applies 5% headroom toward the commit budget. - -**Example 3: SwapTotal ≥ MemTotal (Headroom via ratio not applicable, use overcommit\_kbytes)** - -{% code title="Check memory totals" %} -```bash -cat /proc/meminfo | grep "MemTotal\|SwapTotal" -MemTotal: 16000000 kB -SwapTotal: 16000000 kB -``` -{% endcode %} - -Compute: - -* CommitLimit\_kB = floor(MemTotal × 0.95) = floor(16,000,000 × 0.95) = 15,200,000 kB. - -{% code title="Apply" %} -```bash -echo 2 > /proc/sys/vm/overcommit_memory -echo 15200000 > /proc/sys/vm/overcommit_kbytes -echo 1 > /proc/sys/vm/swappiness -``` -{% endcode %} - -Note that overcommit\_kbytes sets a fixed CommitLimit that ignores swap; recompute if RAM changes. Also note the HugeTLB subtraction does not apply when using overcommit\_kbytes (fixed commit budget). - -Refer to the Linux [proc\_sys\_vm(5)](https://man7.org/linux/man-pages/man5/proc_sys_vm.5.html) manual page for more details on the overcommit\_memory, overcommit\_ratio, and overcommit\_kbytes parameters. - -**Persist Across Reboots** - -To ensure the overcommit remains disabled after reboot, add the three lines below to `/etc/sysctl.conf` (or a file under `/etc/sysctl.d/`). - -{% code title="Add to /etc/sysctl.conf" %} -``` -vm.overcommit_memory = 2 -vm.overcommit_ratio = # if not using overcommit_kbytes -vm.overcommit_kbytes = # if using a fixed CommitLimit -vm.swappiness = 1 -``` -{% endcode %} - -See the Linux [sysctl.conf(5)](https://man7.org/linux/man-pages/man5/sysctl.conf.5.html) manual page for details. - -**NSO Crash Dumps** - -If NSO aborts due to failure to allocate memory, NSO will produce a system dump by default before aborting. When starting NSO from a non-root user, set the `NCS_DUMP` environment variable to point to a filename in a directory that the non-root user can access. The default setting is `NCS_DUMP=ncs_crash.dump`, where the file is written to the NSO run-time directory, typically `NCS_RUN_DIR=/var/opt/ncs`. If the user running NSO cannot write to the directory that the `NCS_DUMP` environment variable points to, generating the system dump file will fail, and the debug information will be lost. - -#### **Alternative: Heuristic Overcommit Mode (vm.overcommit\_memory=0) With Committed\_AS Monitoring** - -As an alternative to the recommended strict mode, `vm.overcommit_memory=2`, you can keep `vm.overcommit_memory=0` to allow overcommit of memory and monitor the total committed memory (Committed\_AS) versus CommitLimit using, for example, a best effort script or observability tool. When Committed\_AS crosses a threshold, for example, 90% of CommitLimit, proactively trigger a series of NSO debug dumps every few seconds via `ncs --debug-dump`. Optionally, a second critical threshold, for example, 95% of CommitLimit, proactively trigger NSO to produce a system dump and then exit gracefully. - -* This approach does not prevent NSO from getting killed; it attempts to capture diagnostic data before memory pressure becomes critical and the Linux OOM-killer kills NSO. -* If swap is enabled, prefer vm.swappiness=1 and consider placing NSO in a cgroup with memory.swap.max=0 to avoid swap I/O for NSO. Requires Linux cgroup v2 and a service-managed cgroup (e.g., systemd) support. - -- Committed\_AS versus CommitLimit is a more meaningful early‑warning signal than Committed\_AS versus MemTotal, because CommitLimit reflects the kernel’s current overcommit policy, swap availability, and huge page reservations—MemTotal does not. -- When in Heuristic mode (vm.overcommit\_memory=0): CommitLimit is informative, not enforced. It’s still better than MemTotal for early warning, but OOM can occur before or after you reach it. -- If necessary for your use-case, complement with MemAvailable, swap activity (vmstat or /proc/vmstat), PSI memory pressure (/proc/pressure/memory), and per‑process/cgroup RSS to catch imminent pressure that Committed\_AS alone may miss. -- Ensure the user running the monitor has permission to execute `ncs --debug-dump` and write to the chosen dump directory. -- See "NSO Crash Dumps" above for crash dump details. - -{% code title="Simple example script NSO debug-dump monitor" overflow="wrap" %} -```bash -#!/usr/bin/env bash -# Simple NSO debug-dump monitor for heuristic overcommit mode (vm.overcommit_memory=0). -# Triggers ncs --debug-dump when Committed_AS reaches 90% of CommitLimit. -# Triggers NSO to produce a system dump before exiting using kill -USR1 when Committed_AS reaches 95% of CommitLimit - -THRESHOLD_PCT=90 # Trigger at 90% of CommitLimit (10% headroom). -CRITICAL_PCT=95 # Trigger at 95% of CommitLimit (5% headroom). -POLL_INTERVAL=5 # Seconds between checks. -PROCESS_CHECK_INTERVAL=30 -DUMP_COUNT=10 # Number of dumps to collect. -DUMP_DELAY=10 # Seconds between dumps. -DUMP_PREFIX="dump" # Files like dump.1.bin, dump.2.bin, ... - -command -v ncs >/dev/null 2>&1 || { echo "ncs command not found in PATH."; exit 1; } - -find_nso_pid() { - pgrep -x ncs.smp | head -n1 || true -} - -while true; do - pid="$(find_nso_pid)" - if [ -z "${pid:-}" ]; then - echo "NSO not running; retry in ${PROCESS_CHECK_INTERVAL}s..." - sleep "$PROCESS_CHECK_INTERVAL" - continue - fi - - committed="$(awk '/Committed_AS:/ {print $2}' /proc/meminfo)" - commit_limit="$(awk '/CommitLimit:/ {print $2}' /proc/meminfo)" - if [ -z "$committed" ] || [ -z "$commit_limit" ]; then - echo "Unable to read /proc/meminfo; retry in ${POLL_INTERVAL}s..." - sleep "$POLL_INTERVAL" - continue - fi - - threshold=$(( commit_limit * THRESHOLD_PCT / 100 )) - critical=$(( commit_limit * CRITICAL_PCT / 100 )) - echo "PID=${pid} Committed_AS=${committed}kB; CommitLimit=${commit_limit}kB; Threshold=${threshold}kB; Critical=${critical}kB." - if [ "$committed" -ge "$critical" ]; then - echo "Critical threshold crossed; collect a system dump and stop NSO..." - kill -USR1 ${pid} - exit 0 - elif [ "$committed" -ge "$threshold" ]; then - echo "Threshold crossed; collecting ${DUMP_COUNT} debug dumps..." - for i in $(seq 1 "$DUMP_COUNT"); do - file="${DUMP_PREFIX}.${i}.bin" - echo "Dump $i -> ${file}" - if ! ncs --debug-dump "$file"; then - echo "Debug dump $i failed." - fi - sleep "$DUMP_DELAY" - done - echo "All debug dumps completed; exiting." - exit 0 - fi - - sleep "$POLL_INTERVAL" -done -``` -{% endcode %} - -
- -{% hint style="info" %} -Some older NSO releases expect the `/etc/init.d/` folder to exist in the host operating system. If the folder does not exist, the installer may fail to successfully install NSO. A workaround that allows the installer to proceed is to create the folder manually, but the NSO process will not automatically start at boot. -{% endhint %} - -### Step 5 - Set Up User Access - -The installation is configured for PAM authentication, with group assignment based on the OS group database (e.g. `/etc/group` file). Users that need access to NSO must belong to either the `ncsadmin` group (for unlimited access rights) or the `ncsoper` group (for minimal access rights). - -To set up user access: - -1. To create the `ncsadmin` group, use the OS shell command: - - ```bash - # groupadd ncsadmin - ``` -2. To create the `ncsoper` group, use the OS shell command: - - ```bash - # groupadd ncsoper - ``` -3. To add an existing user to one of these groups, use the OS shell command: - - ```bash - # usermod -a -G 'groupname' 'username' - ``` - -### Step 6 - Set Environment Variables - -To set environment variables: - -1. Change to Super User privileges. - - ```bash - $ sudo -s - ``` -2. The installation program creates a shell script file in each NSO installation which sets the environment variables needed to run NSO. With the `--system-install` option, by default, these settings are set on the shell. To explicitly set the variables, source `ncs.sh` or `ncs.csh` depending on your shell type. - - ```bash - # source /etc/profile.d/ncs.sh - ``` -3. Start NSO. - - ```bash - # systemctl daemon-reload - # systemctl start ncs - ``` - - NSO starts at boot going forward. - - Once you log on with the user that belongs to `ncsadmin` or `ncsoper`, you can directly access the CLI as shown below: - - ```bash - $ ncs_cli -C - ``` - -### Step 7 - Runtime Directory Creation - -As part of the System Install, the NSO daemon `ncs` is automatically started at boot time. You do not need to create a Runtime Directory for System Install. - -### Step 8 - Generate License Registration Token - -To conclude the NSO installation, a license registration token must be created using a (CSSM) account. This is because NSO uses [Cisco Smart Licensing](../management/system-management/cisco-smart-licensing.md) to make it easy to deploy and manage NSO license entitlements. Login credentials to the [Cisco Smart Software Manager](https://www.cisco.com/c/en/us/buy/smart-accounts/software-manager.html) (CSSM) account are provided by your Cisco contact and detailed instructions on how to [create a registration token](../management/system-management/cisco-smart-licensing.md#d5e2927) can be found in the Cisco Smart Licensing. General licensing information covering licensing models, how licensing works, usage compliance, etc., is covered in the [Cisco Software Licensing Guide](https://www.cisco.com/c/en/us/buy/licensing/licensing-guide.html). - -To generate a license registration token: - -1. When you have a token, start a Cisco CLI towards NSO and enter the token, for example: - - ```bash - $ ncs_cli -Cu admin - admin@ncs# license smart register idtoken - YzIzMDM3MTgtZTRkNC00YjkxLTk2ODQtOGEzMTM3OTg5MG - - Registration process in progress. - Use the 'show license status' command to check the progress and result. - ``` - - \ - Upon successful registration, NSO automatically requests a license entitlement for its own instance and for the number of devices it orchestrates and their NED types. If development mode has been enabled, only development entitlement for the NSO instance itself is requested. -2. Inspect the requested entitlements using the command `show license all` (or by inspecting the NSO daemon log). An example output is shown below. - - ```bash - admin@ncs# show license all - ... - 21-Apr-2016::11:29:18.022 miosaterm confd[8226]: - Smart Licensing Global Notification: - type = "notifyRegisterSuccess", - agentID = "sa1", - enforceMode = "notApplicable", - allowRestricted = false, - failReasonCode = "success", - failMessage = "Successful." - 21-Apr-2016::11:29:23.029 miosaterm confd[8226]: - Smart Licensing Entitlement Notification: type = "notifyEnforcementMode", - agentID = "sa1", - notificationTime = "Apr 21 11:29:20 2016", - version = "1.0", - displayName = "regid.2015-10.com.cisco.NSO-network-element", - requestedDate = "Apr 21 11:26:19 2016", - tag = "regid.2015-10.com.cisco.NSO-network-element", - enforceMode = "inCompliance", - daysLeft = 90, - expiryDate = "Jul 20 11:26:19 2016", - requestedCount = 8 - ... - ``` - -
- -Evaluation Period - -If no registration token is provided, NSO enters a 90-day evaluation period and the remaining evaluation time is recorded hourly in the NSO daemon log: - -``` - ... - 13-Apr-2016::13:22:29.178 miosaterm confd[16260]: -Starting the NCS Smart Licensing Java VM - 13-Apr-2016::13:22:34.737 miosaterm confd[16260]: -Smart Licensing evaluation time remaining: 90d 0h 0m 0s -... - 13-Apr-2016::13:22:34.737 miosaterm confd[16260]: -Smart Licensing evaluation time remaining: 89d 23h 0m 0s -... -``` - -
- -
- -Communication Send Error - -During upgrades, if you experience a 'Communication Send Error' during license registration, restart the Smart Agent. - -
- -
- -If You are Unable to Access Cisco Smart Software Manager - -In a situation where the NSO instance has no direct access to the Cisco Smart Software Manager, one option is the [Cisco Smart Software Manager Satellite](https://software.cisco.com/software/csws/ws/platform/home) which can be installed to manage software licenses on the premises. Install the satellite and use the command `call-home destination address http ` to point to the satellite. - -Another option when direct access is not desired is to configure an HTTP or HTTPS proxy, e.g., `smart-license smart-agent proxy url https://127.0.0.1:8080`. If you plan to do this, take the note below regarding ignored CLI configurations into account: - -If `ncs.conf` contains a configuration for any of the java-executable, java-options, override-url/url, or proxy/url under the configure path `/ncs-config/smart-license/smart-agent/`, then any corresponding configuration done via the CLI is ignored. - -
- -
- -License Registration in HA Mode - -When configuring NSO in High Availability (HA) mode, the license registration token must be provided to the CLI running on the primary node. Read more about HA and node types in [High Availability](../management/high-availability.md)_._ - -
- -
- -Licensing Log - -Licensing activities are also logged in the NSO daemon log as described in [Monitoring NSO](../management/system-management/#d5e7876). For example, a successful token registration results in the following log entry: - -``` - 21-Apr-2016::11:29:18.022 miosaterm confd[8226]: -Smart Licensing Global Notification: -type = "notifyRegisterSuccess" -``` - -
- -
- -Check Registration Status - -To check the registration status, use the command `show license status`. - -```bash -admin@ncs# show license status - -Smart Licensing is ENABLED - -Registration: -Status: REGISTERED -Smart Account: Network Services Orchestrator -Virtual Account: Default -Export-Controlled Functionality: Allowed -Initial Registration: SUCCEEDED on Apr 21 09:29:11 2016 UTC -Last Renewal Attempt: SUCCEEDED on Apr 21 09:29:16 2016 UTC -Next Renewal Attempt: Oct 18 09:29:16 2016 UTC -Registration Expires: Apr 21 09:26:13 2017 UTC -Export-Controlled Functionality: Allowed - -License Authorization: - -License Authorization: -Status: IN COMPLIANCE on Apr 21 09:29:18 2016 UTC -Last Communication Attempt: SUCCEEDED on Apr 21 09:26:30 2016 UTC -Next Communication Attempt: Apr 21 21:29:32 2016 UTC -Communication Deadline: Apr 21 09:26:13 2017 UTC -``` - -
- -## System Install FAQs - -Frequently Asked Questions (FAQs) about System Install. - -
- -Is there a dependency between the NSO Installation Directory and Runtime Directory? - -No, there is no such dependency. - -
- -
- -Do you need to source the ncsrc file before starting NSO? - -No. By default, the environment variables are configured and set on the shell with System Install. - -
- -
- -Can you start NSO from a directory that is not an NSO runtime directory? - -Yes. - -
- -
- -Can you stop NSO from a directory that is not an NSO runtime directory? - -Yes. - -
- -
- -For evaluation and development purposes, instead of a Local Install, you performed a System Install. Now you cannot build or run NSO examples as described in README files. How can you proceed further? - -The easiest way is to uninstall the System install using `ncs-uninstall --all` and do a Local Install from scratch. - -
- -
- -Can we move NSO Installation from one folder to another ? - -No. - -
diff --git a/administration/installation-and-deployment/upgrade-nso.md b/administration/installation-and-deployment/upgrade-nso.md deleted file mode 100644 index 13b05ed4..00000000 --- a/administration/installation-and-deployment/upgrade-nso.md +++ /dev/null @@ -1,501 +0,0 @@ ---- -description: Upgrade NSO to a higher version. ---- - -# Upgrade NSO - -Upgrading the NSO software gives you access to new features and product improvements. Every change carries a risk, and upgrades are no exception. To minimize the risk and make the upgrade process as painless as possible, this section describes the recommended procedures and practices to follow during an upgrade. - -As usual, sufficient preparation avoids many pitfalls and makes the process more straightforward and less stressful. - -## Preparing for Upgrade - -There are multiple aspects that you should consider before starting with the actual upgrade procedure. While the development team tries to provide as much compatibility between software releases as possible, they cannot always avoid all incompatible changes. For example, when a deviation from an RFC standard is found and resolved, it may break clients that depend on the non-standard behavior. For this reason, a distinction is made between maintenance and a major NSO upgrade. - -A maintenance NSO upgrade is within the same branch, i.e., when the first two version numbers stay the same (x.y in the x.y.z NSO version). An example is upgrading from version 6.2.1 to 6.2.2. In the case of a maintenance upgrade, the NSO release contains only corrections and minor enhancements, minimizing the changes. It includes binary compatibility for packages, so there is no need to recompile the .fxs files for a maintenance upgrade. - -Correspondingly, when the first or second number in the version changes, that is called a full or major upgrade. For example, upgrading version 6.3.1 to 6.4 is a major, non-maintenance upgrade. Due to new features, packages must be recompiled, and some incompatibilities could manifest. - -In addition to the above, a package upgrade is when you replace a package with a newer version, such as a NED or a service package. Sometimes, when package changes are not too big, it is possible to supply the new packages as part of the NSO upgrade, but this approach brings additional complexity. Instead, package upgrade and NSO upgrade should in general, be performed as separate actions and are covered as such. - -To avoid surprises during any upgrade, first ensure the following: - -* Hosts have sufficient disk space, as some additional space is required for an upgrade. -* The software is compatible with the target OS. However, sometimes a newer version of Java or system libraries, such as glibc, may be required. -* All the required NEDs and custom packages are compatible with the target NSO version. If you're planning to run the upgraded version in FIPS-compliant mode, make sure to upgrade the NEDs to the latest version. -* Existing packages have been compiled for the new version and are available to you during the upgrade. -* Check whether the existing `ncs.conf` file can be used as-is or needs updating. For example, stronger encryption algorithms may require you to configure additional keying material. -* Review the `CHANGES` file for information on what has changed. -* If upgrading from a no longer supported software version, verify that the upgrade can be performed directly. In situations where the currently installed version is very old, you may have to upgrade to one or more intermediate versions before upgrading to the target version. - -In case it turns out that any of the packages are incompatible or cannot be recompiled, you will need to contact the package developers for an updated or recompiled version. For an official Cisco-supplied package, it is recommended that you always obtain a pre-compiled version if it is available for the target NSO release, instead of compiling the package yourself. - -Additional preparation steps may be required based on the upgrade and the actual setup, such as when using the Layered Service Architecture (LSA) feature. In particular, for a major NSO upgrade in a multi-version LSA cluster, ensure that the new version supports the other cluster members and follow the additional steps outlined in [Deploying LSA](../advanced-topics/layered-service-architecture.md#deploying-lsa) in Layered Service Architecture. - -If you use the High Availability (HA) feature, the upgrade consists of multiple steps on different nodes. To avoid mistakes, you are encouraged to script the process, for which you will need to set up and verify access to all NSO instances with either `ssh`, `nct`, or some other remote management command. For the reference example, we use in this chapter, see [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc). The management station uses shell and Python scripts that use `ssh` to access the Linux shell and NSO CLI and Python Requests for NSO RESTCONF interface access. - -Likewise, NSO 5.3 added support for 256-bit AES encrypted strings, requiring the AES256CFB128 key in the `ncs.conf` configuration. You can generate one with the `openssl rand -hex 32` or a similar command. Alternatively, if you use an external command to provide keys, ensure that it includes a value for an `AES256CFB128_KEY` in the output. - -With regard to init system, NSO 6.4 introduces `systemd` as the default option instead of SysV. In interactive mode, when upgrading to NSO 6.4 and later, the installer prompts the user to continue using the old SysV service or prepare a `systemd` service. In non-interactive mode, a `systemd` service is prepared by default. When using the `--non-interactive` option, the `/etc/systemd/system/ncs.service` file will be overwritten if it already exists. - -Finally, regardless of the upgrade type, ensure that you have a working backup and can easily restore the previous configuration if needed, as described in [Backup and Restore](../management/system-management/#backup-and-restore). - -{% hint style="danger" %} -**Caution** - -The `ncs-backup` (and consequently the `nct backup`) command does not back up the `/opt/ncs/packages` folder. If you make any file changes, back them up separately. - -However, the best practice is not to modify packages in the `/opt/ncs/packages` folder. Instead, if an upgrade requires package recompilation, separate package folders (or files) should be used, one for each NSO version. -{% endhint %} - -## Single Instance Upgrade - -The upgrade of a single NSO instance requires the following steps: - -1. Create a backup. -2. Perform a System Install of the new version. -3. Stop the old NSO server process. -4. Compact the CDB files write log. -5. Update the `/opt/ncs/current` symbolic link. -6. If required, update the `ncs.conf` configuration file. -7. Update the packages in `/var/opt/ncs/packages/` if recompilation is needed. -8. Start the NSO server process, instructing it to reload the packages. - -{% hint style="info" %} -The following steps assume that you are upgrading to the 6.5 release. They pertain to a System Install of NSO, and you must perform them with Super User privileges. - -If you're upgrading from a non-FIPS setup to a [FIPS](https://www.nist.gov/itl/publications-0/federal-information-processing-standards-fips)-compliant setup, ensure that the system requirements comply to FIPS mode install. This entails considering FIPS compliance at OS level as well as configuring NSO to use only FIPS-validated algorithms for keys and certificates. -{% endhint %} - -{% stepper %} -{% step %} -As a best practice, always create a backup before trying to upgrade. - -```bash -# ncs-backup -``` -{% endstep %} - -{% step %} -For the upgrade itself, you must first download to the host and install the new NSO release. At this point, you can choose to install NSO in standard mode or in FIPS mode. - -{% tabs %} -{% tab title="Standard System Install" %} -The standard mode is the regular NSO install and is suitable for most installations. FIPS is disabled in this mode. - -For standard NSO installation, run the installer as below: - -```bash -# sh nso-6.5.linux.x86_64.installer.bin --system-install -``` -{% endtab %} - -{% tab title="FIPS System Install" %} -FIPS mode creates a FIPS-compliant NSO install. - -FIPS mode should only be used for deployments that are subject to strict compliance regulations as the cryptographic functions are then confined to the CiscoSSL FIPS 140-3 module library. - -For FIPS-compliant NSO install, run the installer with the additional `--fips-install` flag. Afterwards, if needed, enable FIPS in `ncs.conf` as described further below. - -```bash -# sh nso-6.5.linux.x86_64.installer.bin --system-install --fips-install -``` -{% endtab %} -{% endtabs %} -{% endstep %} - -{% step %} -Stop the currently running server with the help of `systemd` or an equivalent command relevant to your system. - -```bash -# systemctl stop ncs -Stopping ncs: . -``` -{% endstep %} - -{% step %} -Compact the CDB files write log using, for example, the `ncs --cdb-compact $NCS_RUN_DIR/cdb` command. -{% endstep %} - -{% step %} -Next, you update the symbolic link for the currently selected version to point to the newly installed one, 6.5 in this case. - -```bash -# cd /opt/ncs -# rm -f current -# ln -s ncs-6.5 current -``` -{% endstep %} - -{% step %} -While seldom necessary, at this point, you would also update the `/etc/ncs/ncs.conf` file. If you ran the installer with FIPS mode, update `ncs.conf` accordingly. - -{% hint style="info" %} -**NSO Configuration for FIPS** - -Note the following as part of FIPS-specific configuration: - -1. If you're upgrading from a non-FIPS version (e.g., 6.4) to a FIPS-compliant version (e.g., 6.5), the following `ncs.conf` entry needs to be manually added to enable FIPS. Afterwards, upon upgrading between FIPS-compliant versions, the existing entry automatically updates, eliminating the need for any manual intervention. - -```xml - - true - -``` - -2. Additional environment variables (`NCS_OPENSSL_CONF_INCLUDE`, `NCS_OPENSSL_CONF`, `NCS_OPENSSL_MODULES`) are configured in `ncsrc` for FIPS compliance. -3. The default `crypto.so` is overwritten at install for FIPS compliance. - -Additionally, note that: - -* As certain algorithms typically available with CiscoSSL are not included in the FIPS 140-3 validated module (and therefore disabled in FIPS mode), you need to configure NSO to use only the algorithms and cryptographic suites available through the CiscoSSL FIPS 140-3 object module. -* With FIPS, NSO signals the NEDs to operate in FIPS mode using Bouncy Castle FIPS libraries for Java-based components, ensuring compliance with FIPS 140-3. To support this, NED packages may also require upgrading, as older versions — particularly SSH-based NEDs — often lack the necessary FIPS signaling or Bouncy Castle support required for cryptographic compliance. -* Configure SSH keys in `ncs.conf` and `init.xml`. -{% endhint %} -{% endstep %} - -{% step %} -Now, ensure that the `/var/opt/ncs/packages/` directory has appropriate packages for the new version. It should be possible to continue using the same packages for a maintenance upgrade. But for a major upgrade, you must normally rebuild the packages or use pre-built ones for the new version. You must ensure this directory contains the exact same version of each existing package, compiled for the new release, and nothing else. - -As a best practice, the available packages are kept in `/opt/ncs/packages/` and `/var/opt/ncs/packages/` only contains symbolic links. In this case, to identify the release for which they were compiled, the package file names all start with the corresponding NSO version. Then, you only need to rearrange the symbolic links in the `/var/opt/ncs/packages/` directory. - -```bash -# cd /var/opt/ncs/packages/ -# rm -f * -# for pkg in /opt/ncs/packages/ncs-6.5-*; do ln -s $pkg; done -``` - -{% hint style="warning" %} -Please note that the above package naming scheme is neither required nor enforced. If your package filesystem names differ from it, you will need to adjust the preceding command accordingly. -{% endhint %} -{% endstep %} - -{% step %} -Finally, you start the new version of the NSO server with the `package reload` flag set. Set `NCS_RELOAD_PACKAGES=true` in `/etc/ncs/ncs.systemd.conf` and start NSO: - -```bash -# systemctl start ncs -Starting ncs: ... -``` - -Set the `NCS_RELOAD_PACKAGES` variable in `/etc/ncs/ncs.systemd.conf` back to its previous value or the system would keep performing a packages reload at subsequent starts. - -NSO will perform the necessary data upgrade automatically. However, this process may fail if you have changed or removed any packages. In that case, ensure that the correct versions of all packages are present in `/var/opt/ncs/packages/` and retry the preceding command. - -Also, note that with many packages or data entries in the CDB, this process could take more than 90 seconds and result in the following error message: - -``` -Starting ncs (via systemctl): Job for ncs.service failed -because a timeout was exceeded. See "systemctl status -ncs.service" and "journalctl -xe" for details. [FAILED] -``` - -The above error does not imply that NSO failed to start, just that it took longer than 90 seconds. Therefore, it is recommended you wait some additional time before verifying. -{% endstep %} -{% endstepper %} - -## Recover from a Failed Upgrade - -It is imperative that you have a working copy of data available from which you can restore. That is why you must always create a backup before starting an upgrade. Only a backup guarantees that you can rerun the upgrade or back out of it, should it be necessary. - -The same steps can also be used to restore data on a new, similar host if the OS of the initial host becomes corrupted beyond repair. - -1. First, stop the NSO process if it is running. - - ```bash - # systemctl stop ncs - Stopping ncs: . - ``` -2. Verify and, if necessary, revert the symbolic link in `/opt/ncs/` to point to the initial NSO release. - - ```bash - # cd /opt/ncs - # ls -l current - # ln -s ncs-VERSION current - ``` - - \ - In the exceptional case where the initial version installation was removed or damaged, you will need to re-install it first and redo the step above. -3. Verify if the correct (initial) version of NSO is being used. - - ```bash - # ncs --version - ``` -4. Next, restore the backup. - - ```bash - # ncs-backup --restore - ``` -5. Finally, start the NSO server and verify the restore was successful. - - ```bash - # systemctl start ncs - Starting ncs: . - ``` - -## NSO HA Version Upgrade - -Upgrading NSO in a highly available (HA) setup is a staged process. It entails running various commands across multiple NSO instances at different times. - -The procedure described in this section is used with the rule-based built-in HA clusters. For HA Raft cluster instructions, refer to [Version Upgrade of Cluster Nodes](../management/high-availability.md) in the HA documentation. - -The procedure is almost the same for a maintenance and major NSO upgrade. The difference is that a major upgrade requires the replacement of packages with recompiled ones. Still, a maintenance upgrade is often perceived as easier because there are fewer changes in the product. - -The stages of the upgrade are: - -1. First, enable read-only mode on the designated `primary`, and then on the `secondary` that is enabled for fail-over. -2. Take a full backup on all nodes. -3. If using a 3-node setup, disconnect the 3rd, non-fail-over `secondary` by disabling HA on this node. -4. Disconnect the HA pair by disabling HA on the designated `primary`, temporarily promoting the designated `secondary` to provide the read-only service (and advertise the shared virtual IP address if it is used). -5. Upgrade the designated `primary`. -6. Disable HA on the designated `secondary` node, to allow designated `primary` to become actual `primary` in the next step. -7. Activate HA on the designated `primary`, which will assume its assigned (`primary`) role to provide the full service (and again advertise the shared IP if used). However, at this point, the system is without HA. -8. Upgrade the designated `secondary` node. -9. Activate HA on the designated `secondary`, which will assume its assigned (`secondary`) role, connecting HA again. -10. Verify that HA is operational and has converged. -11. Upgrade the 3rd, non-fail-over `secondary` if it is used, and verify it successfully rejoins the HA cluster. - -Enabling the read-only mode on both nodes is required to ensure the subsequent backup captures the full system state, as well as making sure the `failover-primary` does not start taking writes when it is promoted later on. - -Disabling the non-fail-over `secondary` in a 3-node setup right after taking a backup is necessary when using the built-in HA rule-based algorithm (enabled by default in NSO 5.8 and later). Without it, the node might connect to the `failover-primary` when the failover happens, which disables read-only mode. - -While not strictly necessary, explicitly promoting the designated `secondary` after disabling HA on the `primary` ensures a fast failover, avoiding the automatic reconnection attempts. If using a shared IP solution, such as the Tail-f HCC, this makes sure the shared VIP comes back up on the designated `secondary` as soon as possible. In addition, some older NSO versions do not reset the read-only mode upon disabling HA if they are not acting `primary`. - -Another important thing to note is that all packages used in the upgrade must match the NSO release. If they do not, the upgrade will fail. - -In the case of a major upgrade, you must recompile the packages for the new version. It is highly recommended that you use pre-compiled packages and do not compile them during this upgrade procedure since the compilation can prove nontrivial, and the production hosts may lack all the required (development) tooling. You should use a naming scheme to distinguish between packages compiled for different NSO versions. A good option is for package file names to start with the `ncs-MAJORVERSION-` prefix for a given major NSO version. This ensures multiple packages can co-exist in the `/opt/ncs/packages` folder, and the NSO version they can be used with becomes obvious. - -The following is a transcript of a sample upgrade procedure, showing the commands for each step described above, in a 2-node HA setup, with nodes in their initial designated state. The procedure ensures that this is also the case in the end. - -```xml - -admin@ncs# show high-availability status mode -high-availability status mode primary -admin@ncs# high-availability read-only mode true - - -admin@ncs# show high-availability status mode -high-availability status mode secondary -admin@ncs# high-availability read-only mode true - - -# ncs-backup - - -# ncs-backup - - -admin@ncs# high-availability disable - - -admin@ncs# high-availability be-primary - - -# -# -# systemctl restart ncs -# - - -admin@ncs# high-availability disable - - -admin@ncs# high-availability enable - - -# -# -# systemctl restart ncs -# - - -admin@ncs# high-availability enable -``` - -Scripting is a recommended way to upgrade the NSO version of an HA cluster. The following example script shows the required commands and can serve as a basis for your own customized upgrade script. In particular, the script requires a specific package naming convention above, and you may need to tailor it to your environment. In addition, it expects the new release version and the designated `primary` and `secondary` node addresses as the arguments. The recompiled packages are read from the `packages-MAJORVERSION/` directory. - -For the below example script, we configured our `primary` and `secondary` nodes with their nominal roles that they assume at startup and when HA is enabled. Automatic failover is also enabled so that the `secondary` will assume the `primary` role if the `primary` node goes down. - -{% code title="Configuration on Both Nodes" %} -```xml - - - - n1 - primary - - - n2 - secondary - true - - - true - - true - true - - - - -``` -{% endcode %} - -{% code title="Script for HA Major Upgrade (with Packages)" %} -``` -#!/bin/bash -set -ex - -vsn=$1 -primary=$2 -secondary=$3 -installer_file=nso-${vsn}.linux.x86_64.installer.bin -pkg_vsn=$(echo $vsn | sed -e 's/^\([0-9]\+\.[0-9]\+\).*/\1/') -pkg_dir="packages-${pkg_vsn}" - -function on_primary() { ssh $primary "$@" ; } -function on_secondary() { ssh $secondary "$@" ; } -function on_primary_cli() { ssh -p 2024 $primary "$@" ; } -function on_secondary_cli() { ssh -p 2024 $secondary "$@" ; } - -function upgrade_nso() { - target=$1 - scp $installer_file $target: - ssh $target "sh $installer_file --system-install --non-interactive" - ssh $target "rm -f /opt/ncs/current && \ - ln -s /opt/ncs/ncs-${vsn} /opt/ncs/current" -} -function upgrade_packages() { - target=$1 - do_pkgs=$(ls "${pkg_dir}/" || echo "") - if [ -n "${do_pkgs}" ] ; then - cd ${pkg_dir} - ssh $target 'rm -rf /var/opt/ncs/packages/*' - for p in ncs-${pkg_vsn}-*.gz; do - scp $p $target:/opt/ncs/packages/ - ssh $target "ln -s /opt/ncs/packages/$p /var/opt/ncs/packages/" - done - cd - - fi -} - -# Perform the actual procedure - -on_primary_cli 'request high-availability read-only mode true' -on_secondary_cli 'request high-availability read-only mode true' - -on_primary 'ncs-backup' -on_secondary 'ncs-backup' - -on_primary_cli 'request high-availability disable' -on_secondary_cli 'request high-availability be-primary' -upgrade_nso $primary -upgrade_packages $primary -on_primary `mv /etc/ncs/ncs.systemd.conf /etc/ncs/ncs.systemd.conf.bak' -on_primary 'echo "NCS_RELOAD_PACKAGES=true" > /etc/ncs/ncs.systemd.conf` -on_primary 'systemctl restart ncs' -on_primary `mv /etc/ncs/ncs.systemd.conf.bak /etc/ncs/ncs.systemd.conf' - - -on_secondary_cli 'request high-availability disable' -on_primary_cli 'request high-availability enable' -upgrade_nso $secondary -upgrade_packages $secondary -on_secondary `mv /etc/ncs/ncs.systemd.conf /etc/ncs/ncs.systemd.conf.bak' -on_secondary 'echo "NCS_RELOAD_PACKAGES=true" > /etc/ncs/ncs.systemd.conf` -on_secondary 'systemctl restart ncs' -on_secondary `mv /etc/ncs/ncs.systemd.conf.bak /etc/ncs/ncs.systemd.conf' - -on_secondary_cli 'request high-availability enable' -``` -{% endcode %} - -Once the script is completed, it is paramount that you manually verify the outcome. First, check that the HA is enabled by using the `show high-availability` command on the CLI of each node. Then connect to the designated secondaries and ensure they have the complete latest copy of the data, synchronized from the primaries. - -After the `primary` node is upgraded and restarted, the read-only mode is automatically disabled. This allows the `primary` node to start processing writes, minimizing downtime. However, there is no HA. Should the `primary` fail at this point or you need to revert to a pre-upgrade backup, the new writes would be lost. To avoid this scenario, again enable read-only mode on the `primary` after re-enabling HA. Then disable read-only mode only after successfully upgrading and reconnecting the `secondary`. - -To further reduce time spent upgrading, you can customize the script to install the new NSO release and copy packages beforehand. Then, you only need to switch the symbolic links and restart the NSO process to use the new version. - -You can use the same script for a maintenance upgrade as-is, with an empty `packages-MAJORVERSION` directory, or remove the `upgrade_packages` calls from the script. - -Example implementations that use scripts to upgrade a 2- and 3-node setup using CLI/MAAPI or RESTCONF are available in the NSO example set under [examples.ncs/high-availability](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability). - -We have been using a two-node HCC layer-2 upgrade reference example elsewhere in the documentation to demonstrate installing NSO and adding the initial configuration. The upgrade-l2 example referenced in [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc) implements shell and Python scripted steps to upgrade the NSO version using `ssh` to the Linux shell and the NSO CLI or Python Requests RESTCONF for accessing the `paris` and `london` nodes. See the example for details. - -If you do not wish to automate the upgrade process, you will need to follow the instructions from [Single Instance Upgrade](upgrade-nso.md#ug.admin_guide.manual_upgrade) and transfer the required files to each host manually. Additional information on HA is available in [High Availability](../management/high-availability.md). However, you can run the `high-availability` actions from the preceding script on the NSO CLI as-is. In this case, please take special care of which host you perform each command, as it can be easy to mix them up. - -## Package Upgrade - -Package upgrades are frequent and routine in development but require the same care as NSO upgrades in the production environment. The reason is that the new packages may contain an updated YANG model, resulting in a data upgrade process similar to a version upgrade. So, if a package is removed or uninstalled and a replacement is not provided, package-specific data, such as service instance data, will also be removed. - -In a single-node environment, the procedure is straightforward. Create a backup with the `ncs-backup` command and ensure the new package is compiled for the current NSO version and available under the `/opt/ncs/packages` directory. Then either manually rearrange the symbolic links in the `/var/opt/ncs/packages` directory or use the `software packages install` command in the NSO CLI. Finally, invoke the `packages reload` command. For example: - -```bash -# ncs-backup -INFO Backup /var/opt/ncs/backups/ncs-6.4@2024-04-21T10:34:42.backup.gz created -successfully -# ls /opt/ncs/packages -ncs-6.4-router-nc-1.0 ncs-6.4-router-nc-1.0.2 -# ncs_cli -C -admin@ncs# software packages install package router-nc-1.0.2 replace-existing -installed ncs-6.4-router-nc-1.0.2 -admin@ncs# packages reload - ->>> System upgrade is starting. ->>> Sessions in configure mode must exit to operational mode. ->>> No configuration changes can be performed until upgrade has completed. ->>> System upgrade has completed successfully. -reload-result { - package router-nc-1.0.2 - result true -} -``` - -On the other hand, upgrading packages in an HA setup is an error-prone process. Thus, NSO provides an action, `packages ha sync and-reload`to minimize such complexity. It is considerably faster and more efficient than upgrading one node at a time. - -{% hint style="info" %} -If the only change in the packages is the addition of new NED packages, the `and-add` can replace `and-reload` command for an even more optimized and less intrusive update. See [Adding NED Packages](../management/package-mgmt.md#ug.package_mgmt.ned_package_add) for details. -{% endhint %} - -The action executes on the `primary` node. First, it syncs the physical packages from the `primary` node to the `secondary` nodes as tar archive files, regardless if the packages were initially added as directories or tar archives. Then, it performs the upgrade on all nodes in one go. The action does not sync packages to or upgrade nodes with the `none` role. - -The `packages ha sync` action only distributes new packages to the _secondary_ nodes. If a package already exists on the `secondary` node, it will replace it with the one on the `primary` node. Deleting a package on the `primary` node will also delete it on the `secondary` node. Packages found in load paths under the installation destination (by default `/opt/ncs/current`) are not distributed as they belong to the system and should not differ between the `primary` and the `secondary` nodes. - -It is crucial to ensure that the load path configuration is identical on both `primary` and `secondary` nodes. Otherwise, the distribution will not start, and the action output will contain detailed error information. - -Using the `and-reload` parameter with the action starts the upgrade once packages are copied over. The action sets the `primary` node to read-only mode. After the upgrade is successfully completed, the node is set back to its previous mode. - -If the parameter `and-reload` is also supplied with the `wait-commit-queue-empty` parameter, it will wait for the commit queue to become empty on the `primary` node and prevent other queue items from being added while the queue is being drained. - -Using the `wait-commit-queue-empty` parameter is the recommended approach, as it minimizes the risk of the upgrade failing due to commit queue items still relying on the old schema. - -{% code title="Package Upgrade Procedure" %} -```bash -primary@node1# software packages list -package { - name dummy-1.0.tar.gz - loaded -} -primary@node1# software packages fetch package-from-file \ -$MY_PACKAGE_STORE/dummy-1.1.tar.gz -primary@node1# software packages install package dummy-1.1 replace-existing -primary@node1# packages ha sync and-reload { wait-commit-queue-empty } -``` -{% endcode %} - -The `packages ha sync and-reload` command has the following known limitations and side effects: - -* The `primary` node is set to `read-only` mode before the upgrade starts, and it is set back to its previous mode if the upgrade is successfully upgraded. However, the node will always be in read-write mode if an error occurs during the upgrade. It is up to the user to set the node back to the desired mode by using the `high-availability read-only mode` command. -* As a best practice, you should create a backup of all nodes before upgrading. This action creates no backups, you must do that explicitly. - -Example implementations that use scripts to upgrade a 2- and 3-node setup using CLI/MAAPI or RESTCONF are available in the NSO example set under [examples.ncs/high-availability](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availabilit). - -We have been using a two-node HCC layer 2 upgrade reference example elsewhere in the documentation to demonstrate installing NSO and adding the initial configuration. The `upgrade-l2` example referenced in [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc) implements shell and Python scripted steps to upgrade the `primary` `paris` package versions and sync the packages to the `secondary` `london` using `ssh` to the Linux shell and the NSO CLI or Python Requests RESTCONF for accessing the `paris` and `london` nodes. See the example for details. - -In some cases, NSO may warn when the upgrade looks suspicious. For more information on this, see [Loading Packages](../management/package-mgmt.md#ug.package_mgmt.loading). If you understand the implications and are willing to risk losing data, use the `force` option with `packages reload` or set the `NCS_RELOAD_PACKAGES` environment variable to `force` when restarting NSO. It will force NSO to ignore warnings and proceed with the upgrade. In general, this is not recommended. - -In addition, you must take special care of NED upgrades because services depend on them. For example, since NSO 5 introduced the CDM feature, which allows loading multiple versions of a NED, a major NED upgrade requires a procedure involving the `migrate` action. - -When a NED contains nontrivial YANG model changes, that is called a major NED upgrade. The NED ID changes, and the first or second number in the NED version changes since NEDs follow the same versioning scheme as NSO. In this case, you cannot simply replace the package, as you would for a maintenance or patch NED release. Instead, you must load (add) the new NED package alongside the old one and perform the migration. - -Migration uses the `/ncs:devices/device/migrate` action to change the ned-id of a single device or a group of devices. It does not affect the actual network device, except possibly reading from it. So, the migration does not have to be performed as part of the package upgrade procedure described above but can be done later, during normal operations. The details are described in [NED Migration](../management/ned-administration.md#sec.ned_migration). Once the migration is complete, you can remove the old NED by performing another package upgrade, where you deinstall the old NED package. It can be done straight after the migration or as part of the next upgrade cycle. diff --git a/administration/management/README.md b/administration/management/README.md deleted file mode 100644 index 49e72dfa..00000000 --- a/administration/management/README.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -description: Perform system management tasks on your NSO deployment. -icon: folder-gear ---- - -# Management - diff --git a/administration/management/aaa-infrastructure.md b/administration/management/aaa-infrastructure.md deleted file mode 100644 index fe2c53ba..00000000 --- a/administration/management/aaa-infrastructure.md +++ /dev/null @@ -1,1473 +0,0 @@ ---- -description: >- - Manage user authentication, authorization, and audit using NSO's AAA - mechanism. ---- - -# AAA Infrastructure - -## The Problem - -Users log into NSO through the CLI, NETCONF, RESTCONF, SNMP, or via the Web UI. In either case, users need to be authenticated. That is, a user needs to present credentials, such as a password or a public key to gain access. As an alternative, for RESTCONF, users can be authenticated via token validation. - -Once a user is authenticated, all operations performed by that user need to be authorized. That is, certain users may be allowed to perform certain tasks, whereas others are not. This is called authorization. We differentiate between the authorization of commands and the authorization of data access. - -## Structure - Data Models - -The NSO daemon manages device configuration including AAA information. NSO manages AAA information as well as uses it. The AAA information describes which users may log in, what passwords they have, and what they are allowed to do. This is solved in NSO by requiring a data model to be both loaded and populated with data. NSO uses the YANG module `tailf-aaa.yang` for authentication, while `ietf-netconf-acm.yang` (NETCONF Access Control Model (NACM), [RFC 8341](https://tools.ietf.org/html/rfc8341)) as augmented by `tailf-acm.yang` is used for group assignment and authorization. - -### Data Model Contents - -The NACM data model is targeted specifically towards access control for NETCONF operations and thus lacks some functionality that is needed in NSO, in particular, support for the authorization of CLI commands and the possibility to specify the context (NETCONF, CLI, etc.) that a given authorization rule should apply to. This functionality is modeled by augmentation of the NACM model, as defined in the `tailf-acm.yang` YANG module. - -The `ietf-netconf-acm.yang` and `tailf-acm.yang` modules can be found in `$NCS_DIR/src/ncs/yang` directory in the release, while `tailf-aaa.yang` can be found in the `$NCS_DIR/src/ncs/aaa` directory. - -NACM options related to services are modeled by augmentation of the NACM model, as defined in the `tailf-ncs-acm.yang` YANG module. The `tailf-ncs-acm.yang` can be found in `$NCS_DIR/src/ncs/yang` directory in the release. - -The complete AAA data model defines a set of users, a set of groups, and a set of rules. The data model must be populated with data that is subsequently used by by NSO itself when it authenticates users and authorizes user data access. These YANG modules work exactly like all other `fxs` files loaded into the system with the exception that NSO itself uses them. The data belongs to the application, but NSO itself is the user of the data. - -Since NSO requires a data model for the AAA information for its operation, it will report an error and fail to start if these data models cannot be found. - -## AAA-related Items in `ncs.conf` - -NSO itself is configured through a configuration file - `ncs.conf`. In that file, we have the following items related to authentication and authorization: - -* `/ncs-config/aaa/ssh-server-key-dir`: If SSH termination is enabled for NETCONF or the CLI, the NSO built-in SSH server needs to have server keys. These keys are generated by the NSO install script and by default end up in `$NCS_DIR/etc/ncs/ssh`.\ - \ - It is also possible to use OpenSSH to terminate NETCONF or the CLI. If OpenSSH is used to terminate SSH traffic, this setting has no effect. -* `/ncs-config/aaa/ssh-pubkey-authentication`: If SSH termination is enabled for NETCONF or the CLI, this item controls how the NSO SSH daemon locates the user keys for public key authentication. See [Public Key Login](aaa-infrastructure.md#ug.aaa.public_key_login) for details. -* `/ncs-config/aaa/local-authentication/enabled`: The term 'local user' refers to a user stored under `/aaa/authentication/users`. The alternative is a user unknown to NSO, typically authenticated by PAM. By default, NSO first checks local users before trying PAM or external authentication.\ - \ - Local authentication is practical in test environments. It is also useful when we want to have one set of users that are allowed to log in to the host with normal shell access and another set of users that are only allowed to access the system using the normal encrypted, fully authenticated, northbound interfaces of NSO.\ - \ - If we always authenticate users through PAM, it may make sense to set this configurable to `false`. If we disable local authentication, it implicitly means that we must use either PAM authentication or external authentication. It also means that we can leave the entire data trees under `/aaa/authentication/users` and, in the case of external authentication, also `/nacm/groups` (for NACM) or `/aaa/authentication/groups` (for legacy tailf-aaa) empty. -* `/ncs-config/aaa/pam`: NSO can authenticate users using PAM (Pluggable Authentication Modules). PAM is an integral part of most Unix-like systems.\ - \ - PAM is a complicated - albeit powerful - subsystem. It may be easier to have all users stored locally on the host, However, if we want to store users in a central location, PAM can be used to access the remote information. PAM can be configured to perform most login scenarios including RADIUS and LDAP. One major drawback with PAM authentication is that there is no easy way to extract the group information from PAM. PAM authenticates users, it does not also assign a user to a set of groups. PAM authentication is thoroughly described later in this chapter. -* `/ncs-config/aaa/default-group`: If this configuration parameter is defined and if the group of a user cannot be determined, a logged-in user ends up in the given default group. -* `/ncs-config/aaa/external-authentication`: NSO can authenticate users using an external executable. This is further described later in [External Authentication](aaa-infrastructure.md#ug.aaa.external_authentication). As an alternative, you may consider using package authentication. -* `/ncs-config/aaa/external-validation`: NSO can authenticate users by validation of tokens using an external executable. This is further described later in [External Token Validation](aaa-infrastructure.md#ug.aaa.external_validation). Where external authentication uses a username and password to authenticate a user, external validation uses a token. The validation script should use the token to authenticate a user and can, optionally, also return a new token to be returned with the result of the request. It is currently only supported for RESTCONF. -* `/ncs-config/aaa/external-challenge`: NSO has support for multi-factor authentication by sending challenges to a user. Challenges may be sent from any of the external authentication mechanisms but are currently only supported by JSON-RPC and CLI over SSH. This is further described later in [External Multi-factor Authentication](aaa-infrastructure.md#ug.aaa.external_challenge). -* `/ncs-config/aaa/package-authentication`: NSO can authenticate users using package authentication. It extends the concept of external authentication by allowing multiple packages to be used for authentication instead of a single executable. This is further described in [Package Authentication](aaa-infrastructure.md#ug.aaa.packageauth). -* `/ncs-config/aaa/single-sign-on`: With this setting enabled, NSO invokes Package Authentication on all requests to HTTP endpoints with the `/sso` prefix. This way, Package Authentication packages that require custom endpoints can expose them under the `/sso` base route.\ - \ - For example, a SAMLv2 Single Sign-On (SSO) package needs to process requests to an AssertionConsumerService endpoint, such as `/sso/saml/acs`, and therefore requires enabling this setting.\ - \ - This is a valid authentication method for WEB UI and JSON-RPC interfaces and needs Package Authentication to be enabled as well. -* `/ncs-config/aaa/single-sign-on/enable-automatic-redirect`: If only one Single Sign-On package is configured (a package with `single-sign-on-url` set in `package-meta-data.xml`) and also this setting is enabled, NSO automatically redirects all unauthenticated access attempts to the configured `single-sign-on-url`. - -## Authentication - -Depending on the northbound management protocol, when a user session is created in NSO, it may or may not be authenticated. If the session is not yet authenticated, NSO's AAA subsystem is used to perform authentication and authorization, as described below. If the session already has been authenticated, NSO's AAA assigns groups to the user as described in [Group Membership](aaa-infrastructure.md#ug.aaa.groups), and performs authorization, as described in [Authorization](aaa-infrastructure.md#ug.aaa.authorization). - -The authentication part of the data model can be found in `tailf-aaa.yang`: - -```yang - container authentication { - tailf:info "User management"; - container users { - tailf:info "List of local users"; - list user { - key name; - leaf name { - type string; - tailf:info "Login name of the user"; - } - leaf uid { - type int32; - mandatory true; - tailf:info "User Identifier"; - } - leaf gid { - type int32; - mandatory true; - tailf:info "Group Identifier"; - } - leaf password { - type passwdStr; - mandatory true; - } - leaf ssh_keydir { - type string; - mandatory true; - tailf:info "Absolute path to directory where user's ssh keys - may be found"; - } - leaf homedir { - type string; - mandatory true; - tailf:info "Absolute path to user's home directory"; - } - } - } - } -``` - -AAA authentication is used in the following cases: - -* When the built-in SSH server is used for NETCONF and CLI sessions. -* For Web UI sessions and REST access. -* When the method `Maapi.Authenticate()` is used. - -NSO's AAA authentication is not used in the following cases: - -* When NETCONF uses an external SSH daemon, such as OpenSSH. - - \ - In this case, the NETCONF session is initiated using the program `netconf-subsys`, as described in [NETCONF Transport Protocols](../../development/core-concepts/northbound-apis/#ug.netconf_agent.transport) in Northbound APIs. -* When NETCONF uses TCP, as described in [NETCONF Transport Protocols](../../development/core-concepts/northbound-apis/#ug.netconf_agent.transport) in Northbound APIs, e.g. through the command `netconf-console`. -* When accessing the CLI by invoking the `ncs_cli`, e.g. through an external SSH daemon, such as OpenSSH, or a telnet daemon.\ - \ - An important special case here is when a user has shell access to the host and runs **ncs\_cli** from the shell. This command, as well as direct access to the IPC socket, allows for authentication bypass. It is crucial to consider this case for your deployment. If non-trusted users have shell access to the host, IPC access must be restricted. See [Authenticating IPC Access](aaa-infrastructure.md#authenticating-ipc-access). -* When SNMP is used, SNMP has its own authentication mechanisms. See [NSO SNMP Agent](../../development/core-concepts/northbound-apis/#the-nso-snmp-agent) in Northbound APIs. -* When the method `Maapi.startUserSession()` is used without a preceding call of `Maapi.authenticate()`. - -### Public Key Login - -When a user logs in over NETCONF or the CLI using the built-in SSH server, with a public key login, the procedure is as follows. - -The user presents a username in accordance with the SSH protocol. The SSH server consults the settings for `/ncs-config/aaa/ssh-pubkey-authentication` and `/ncs-config/aaa/local-authentication/enabled` . - -1. If `ssh-pubkey-authentication` is set to `local`, and the SSH keys in `/aaa/authentication/users/user{$USER}/ssh_keydir` match the keys presented by the user, authentication succeeds. -2. Otherwise, if `ssh-pubkey-authentication` is set to `system`, `local-authentication` is enabled, and the SSH keys in `/aaa/authentication/users/user{$USER}/ssh_keydir` match the keys presented by the user, authentication succeeds. -3. Otherwise, if `ssh-pubkey-authentication` is set to `system` and the user `/aaa/authentication/users/user{$USER}` does not exist, but the user does exist in the OS password database, the keys in the user's `$HOME/.ssh` directory are checked. If these keys match the keys presented by the user, authentication succeeds. -4. Otherwise, authentication fails. - -In all cases the keys are expected to be stored in a file called `authorized_keys` (or `authorized_keys2` if `authorized_keys` does not exist), and in the native OpenSSH format (i.e. as generated by the OpenSSH `ssh-keygen` command). If authentication succeeds, the user's group membership is established as described in [Group Membership](aaa-infrastructure.md#ug.aaa.groups). - -This is exactly the same procedure that is used by the OpenSSH server with the exception that the built-in SSH server also may locate the directory containing the public keys for a specific user by consulting the `/aaa/authentication/users` tree. - -### **Setting up Public Key Login** - -We need to provide a directory where SSH keys are kept for a specific user and give the absolute path to this directory for the `/aaa/authentication/users/user/ssh_keydir` leaf. If a public key login is not desired at all for a user, the value of the `ssh_keydir` leaf should be set to `""`, i.e. the empty string. Similarly, if the directory does not contain any SSH keys, public key logins for that user will be disabled. - -The built-in SSH daemon supports DSA, RSA, and ED25519 keys. To generate and enable RSA keys of size 4096 bits for, say, user "bob", the following steps are required. - -On the client machine, as user "bob", generate a private/public key pair as: - -```bash -# ssh-keygen -b 4096 -t rsa -Generating public/private rsa key pair. -Enter file in which to save the key (/home/bob/.ssh/id_rsa): -Created directory '/home/bob/.ssh'. -Enter passphrase (empty for no passphrase): -Enter same passphrase again: -Your identification has been saved in /home/bob/.ssh/id_rsa. -Your public key has been saved in /home/bob/.ssh/id_rsa.pub. -The key fingerprint is: -ce:1b:63:0a:f9:d4:1d:04:7a:1d:98:0c:99:66:57:65 bob@buzz -# ls -lt ~/.ssh -total 8 --rw------- 1 bob users 3247 Apr 4 12:28 id_rsa --rw-r--r-- 1 bob users 738 Apr 4 12:28 id_rsa.pub -``` - -Now we need to copy the public key to the target machine where the NETCONF or CLI SSH client runs. - -Assume we have the following user entry: - -```xml - - bob - 100 - 10 - $1$feedbabe$nGlMYlZpQ0bzenyFOQI3L1 - /var/system/users/bob/.ssh - /var/system/users/bob - -``` - -We need to copy the newly generated file `id_rsa.pub`, which is the public key, to a file on the target machine called `/var/system/users/bob/.ssh/authorized_keys`. - -{% hint style="info" %} -Since the release of [OpenSSH 7.0](https://www.openssh.com/txt/release-7.0), support of `ssh-dss` host and user keys is disabled by default. If you want to continue using these, you may re-enable it using the following options for OpenSSH client: - -``` -HostKeyAlgorithms=+ssh-dss -PubkeyAcceptedKeyTypes=+ssh-dss -``` - -You can find full instructions at [OpenSSH Legacy Options](https://www.openssh.com/legacy.html) webpage. -{% endhint %} - -### Password Login - -Password login is triggered in the following cases: - -* When a user logs in over NETCONF or the CLI using the built-in SSH server, with a password. The user presents a username and a password in accordance with the SSH protocol. -* When a user logs in using the Web UI. The Web UI asks for a username and password. -* When the method `Maapi.authenticate()` is used. - -In this case, NSO will by default try local authentication, PAM, external authentication, and package authentication in that order, as described below. It is possible to change the order in which these are tried, by modifying the `ncs.conf`. parameter `/ncs-config/aaa/auth-order`. See [ncs.conf(5)](../../resources/man/ncs.conf.5.md) in Manual Pages for details. - -1. If `/aaa/authentication/users/user{$USER}` exists and the presented password matches the encrypted password in `/aaa/authentication/users/user{$USER}/password`, the user is authenticated. -2. If the password does not match or if the user does not exist in `/aaa/authentication/users`, PAM login is attempted, if enabled. See [PAM](aaa-infrastructure.md#ug.aaa.pam) for details. -3. If all of the above fails and external authentication is enabled, the configured executable is invoked. See [External Authentication](aaa-infrastructure.md#ug.aaa.external_authentication) for details. - -If authentication succeeds, the user's group membership is established as described in [Group Membership](aaa-infrastructure.md#ug.aaa.groups). - -### PAM - -On operating systems supporting PAM, NSO also supports PAM authentication. Using PAM, authentication with NSO can be very convenient since it allows us to have the same set of users and groups having access to NSO as those that have access to the UNIX/Linux host itself. - -{% hint style="info" %} -PAM is the recommended way to authenticate NSO users. -{% endhint %} - -If we use PAM, we do not have to have any users or any groups configured in the NSO aaa namespace at all. - -To configure PAM we typically need to do the following: - -1. Remove all users and groups from the AAA initialization XML file. -2. Enable PAM in `ncs.conf` by adding the following to the AAA section in `ncs.conf`. The `service` name specifies the PAM service, typically a file in the directory `/etc/pam.d`, but may alternatively, be an entry in a file `/etc/pam.conf` depending on OS and version. Thus, it is possible to have a different login procedure for NSO than for the host itself. - - ```xml - - true - common-auth - - ``` -3. If PAM is enabled and we want to use PAM for login, the system may have to run as `root`. This depends on how PAM is configured locally. However, the default system authentication will typically require `root`, since the PAM libraries then read `/etc/shadow`. If we don't want to run NSO as root, the solution here is to change the owner of a helper program called `$NCS_DIR/lib/ncs/lib/pam-*/priv/epam` and also set the `setuid` bit. - - ```bash - # cd $NCS_DIR/lib/ncs/lib/pam-*/priv/ - # chown root:root epam - # chmod u+s epam - ``` - -As an example, say that we have a user test in `/etc/passwd`, and furthermore: - -```bash -# grep test /etc/group -operator:x:37:test -admin:x:1001:test -``` - -Thus, the `test` user is part of the `admin` and the `operator` groups and logging in to NSO as the `test` user through CLI SSH, Web UI, or NETCONF, renders the following in the audit log. - -``` - 28-Jan-2009::16:05:55.663 buzz ncs[14658]: audit user: test/0 logged - in over ssh from 127.0.0.1 with authmeth:password - 28-Jan-2009::16:05:55.670 buzz ncs[14658]: audit user: test/5 assigned - to groups: operator,admin - 28-Jan-2009::16:05:57.655 buzz ncs[14658]: audit user: test/5 CLI 'exit' -``` - -Thus, the `test` user was found and authenticated from `/etc/passwd`, and the crucial group assignment of the test user was done from `/etc/group`. - -If we wish to be able to also manipulate the users, their passwords, etc on the device, we can write a private YANG model for that data, store that data in CDB, set up a normal CDB subscriber for that data, and finally when our private user data is manipulated, our CDB subscriber picks up the changes and changes the contents of the relevant `/etc` files. - -### External Authentication - -A common situation is when we wish to have all authentication data stored remotely, not locally, for example on a remote RADIUS or LDAP server. This remote authentication server typically not only stores the users and their passwords but also the group information. - -If we wish to have not only the users but also the group information stored on a remote server, the best option for NSO authentication is to use external authentication. - -If this feature is configured, NSO will invoke the executable configured in `/ncs-config/aaa/external-authentication/executable` in `ncs.conf` , and pass the username and the clear text password on `stdin` using the string notation: `"[user;password;]\n"`. - -For example, if the user `bob` attempts to log in over SSH using the password 'secret', and external authentication is enabled, NSO will invoke the configured executable and write `"[bob;secret;]\n"` on the `stdin` stream for the executable. The task of the executable is then to authenticate the user and also establish the username-to-groups mapping. - -For example, the executable could be a RADIUS client which utilizes some proprietary vendor attributes to retrieve the groups of the user from the RADIUS server. If authentication is successful, the program should write `accept` followed by a space-separated list of groups that the user is a member of, and additional information as described below. Again, assuming that bob's password indeed was 'secret', and that bob is a member of the `admin` and the `lamers` groups, the program should write `accept admin lamers $uid $gid $supplementary_gids $HOME` on its standard output and then exit. - -{% hint style="info" %} -There is a general limit of 16000 bytes of output from the `externalauth` program. -{% endhint %} - -Thus, the format of the output from an `externalauth` program when authentication is successful should be: - -**`"accept $groups $uid $gid $supplementary_gids $HOME\n"`** - -Where: - -* `$groups` is a space-separated list of the group names the user is a member of. -* `$uid` is the UNIX integer user ID that NSO should use as a default when executing commands for this user. -* `$gid` is the UNIX integer group ID that NSO should use as a default when executing commands for this user. -* `$supplementary_gids` is a (possibly empty) space-separated list of additional UNIX group IDs the user is also a member of. -* `$HOME` is the directory that should be used as HOME for this user when NSO executes commands on behalf of this user. - -It is further possible for the program to return a token on successful authentication, by using `"accept_token"` instead of `"accept"`: - -**`"accept_token $groups $uid $gid $supplementary_gids $HOME $token\n"`** - -Where: - -* `$token` is an arbitrary string. NSO will then, for some northbound interfaces, include this token in responses. - -It is also possible for the program to return additional information on successful authentication, by using `"accept_info"` instead of `"accept"`: - -**`"accept_info $groups $uid $gid $supplementary_gids $HOME $info\n"`** - -Where: - -* `$info` is some arbitrary text. NSO will then just append this text to the generated audit log message (CONFD\_EXT\_LOGIN). - -Yet another possibility is for the program to return a warning that the user's password is about to expire, by using `"accept_warning"` instead of `"accept"`: - -**`"accept_warning $groups $uid $gid $supplementary_gids $HOME $warning\n"`** - -Where: - -* `$warning` is an appropriate warning message. The message will be processed by NSO according to the setting of `/ncs-config/aaa/expiration-warning` in `ncs.conf`. - -There is also support for token variations of `"accept_info"` and `"accept_warning"` namely `"accept_token_info"` and `"accept_token_warning"`. Both `"accept_token_info"` and `"accept_token_warning"` expect the external program to output exactly the same as described above with the addition of a token after `$HOME`: - -* `"accept_token_info $groups $uid $gid $supplementary_gids $HOME $token $info\n"` -* `"accept_token_warning $groups $uid $gid $supplementary_gids $HOME $token $warning\n"` - -If authentication failed, the program should write `"reject"` or `"abort"`, possibly followed by a reason for the rejection, and a trailing newline. For example, `"reject Bad password\n"` or just `"abort\n"`. The difference between `"reject"` and `"abort"` is that with `"reject"`, NSO will try subsequent mechanisms configured for `/ncs-config/aaa/auth-order` in `ncs.conf` (if any), while with `"abort"`, the authentication fails immediately. Thus `"abort"` can prevent subsequent mechanisms from being tried, but when external authentication is the last mechanism (as in the default order), it has the same effect as `"reject"`. - -Supported by some northbound APIs, such as JSON-RPC and CLI over SSH, the external authentication may also choose to issue a challenge: - -`"challenge $challenge-id $challenge-prompt\n"` - -{% hint style="info" %} -The challenge-prompt may be multi-line, why it must be base64 encoded. -{% endhint %} - -For more information on multi-factor authentication, see [External Multi-Factor Authentication](aaa-infrastructure.md#ug.aaa.external_challenge). - -When external authentication is used, the group list returned by the external program is prepended by any possible group information stored locally under the `/aaa` tree. Hence when we use external authentication it is indeed possible to have the entire `/aaa/authentication` tree empty. The group assignment performed by the external program will still be valid and the relevant groups will be used by NSO when the authorization rules are checked. - -### External Token Validation - -When username and password authentication is not feasible, authentication by token validation is possible. Currently, only RESTCONF supports this mode of authentication. It shares all properties of external authentication, but instead of a username and password, it takes a token as input. The output is also almost the same, the only difference is that it is also expected to output a username. - -If this feature is configured, NSO will invoke the executable configured in `/ncs-config/aaa/external-validation/executable` in `ncs.conf` , and pass the token on `stdin` using the string notation: `"[token;]\n"`. - -For example if the user `bob` attempts to log over RESTCONF using the token `topsecret`, and external validation is enabled, NSO will invoke the configured executable and write `"[topsecret;]\n"` on the `stdin` stream for the executable. - -The task of the executable is then to validate the token, thereby authenticating the user and also establishing the username and username-to-groups mapping. - -For example, the executable could be a FUSION client that utilizes some proprietary vendor attributes to retrieve the username and groups of the user from the FUSION server. If token validation is successful, the program should write `accept` followed by a space-separated list of groups that the user is a member of, and additional information as described below. Again, assuming that `bob`'s token indeed was `topsecret`, and that `bob` is a member of the `admin` and the `lamers` groups, the program should write `accept admin lamers $uid $gid $supplementary_gids $HOME $USER` on its standard output and then exit. - -{% hint style="info" %} -There is a general limit of 16000 bytes of output from the `externalvalidation` program. -{% endhint %} - -Thus the format of the output from an `externalvalidation` program when token validation authentication is successful should be: - -`"accept $groups $uid $gid $supplementary_gids $HOME $USER\n"` - -Where: - -* `$groups` is a space-separated list of the group names the user is a member of. -* `$uid` is the UNIX integer user ID NSO should use as a default when executing commands for this user. -* `$gid` is the UNIX integer group ID NSO should use as a default when executing commands for this user. -* `$supplementary_gids` is a (possibly empty) space-separated list of additional UNIX group IDs the user is also a member of. -* `$HOME` is the directory that should be used as HOME for this user when NSO executes commands on behalf of this user. -* `$USER` is the user derived from mapping the token. - -It is further possible for the program to return a new token on successful token validation authentication, by using `"accept_token"` instead of `"accept"`: - -`"accept_token $groups $uid $gid $supplementary_gids $HOME $USER $token\n"` - -Where: - -* `$token` is an arbitrary string. NSO will then, for some northbound interfaces, include this token in responses. - -It is also possible for the program to return additional information on successful token validation authentication, by using `"accept_info"` instead of `"accept"`: - -`"accept_info $groups $uid $gid $supplementary_gids $HOME $USER $info\n"` - -Where: - -* `$info` is some arbitrary text. NSO will then just append this text to the generated audit log message (CONFD\_EXT\_LOGIN). - -Yet another possibility is for the program to return a warning that the user's password is about to expire, by using `"accept_warning"` instead of `"accept"`: - -`"accept_warning $groups $uid $gid $supplementary_gids $HOME $USER $warning\n"` - -Where: - -* `$warning` is an appropriate warning message. The message will be processed by NSO according to the setting of `/ncs-config/aaa/expiration-warning` in `ncs.conf`. - -There is also support for token variations of `"accept_info"` and `"accept_warning"` namely `"accept_token_info"` and `"accept_token_warning"`. Both `"accept_token_info"` and `"accept_token_warning"` expect the external program to output exactly the same as described above with the addition of a token after `$USER`: - -`"accept_token_info $groups $uid $gid $supplementary_gids $HOME $USER $token $info\n"` - -`"accept_token_warning $groups $uid $gid $supplementary_gids $HOME $USER $token $warning\n"` - -If token validation authentication fails, the program should write `"reject"` or `"abort"`, possibly followed by a reason for the rejection and a trailing newline. For example `"reject Bad password\n"` or just `"abort\n"`. The difference between `"reject"` and `"abort"` is that with `"reject"`, NSO will try subsequent mechanisms configured for `/ncs-config/aaa/validation-order` in `ncs.conf` (if any), while with `"abort"`, the token validation authentication fails immediately. Thus `"abort"` can prevent subsequent mechanisms from being tried. Currently, the only available token validation authentication mechanism is the external one. - -Supported by some northbound APIs, such as JSON-RPC and CLI over SSH, the external validation may also choose to issue a challenge: - -`"challenge $challenge-id $challenge-prompt\n"` - -{% hint style="info" %} -The challenge prompt may be multi-line, why it must be base64 encoded. -{% endhint %} - -For more information on multi-factor authentication, see [External Multi-Factor Authentication](aaa-infrastructure.md#ug.aaa.external_challenge). - -### External Multi-Factor Authentication - -When username, password, or token authentication is not enough, a challenge may be sent from any of the external authentication mechanisms to the user. A challenge consists of a challenge ID and a base64 encoded challenge prompt, and a user is supposed to send a response to the challenge. Currently, only JSONRPC and CLI over SSH support multi-factor authentication. Responses to challenges of multi-factor authentication have the same output as the token authentication mechanism. - -If this feature is configured, NSO will invoke the executable configured in `/ncs-config/aaa/external-challenge/executable` in `ncs.conf` , and pass the challenge ID and response on `stdin` using the string notation: `"[challenge-id;response;]\n"`. - -For example, a user `bob` has received a challenge from external authentication, external validation, or external challenge and then attempts to log in over JSON-RPC with a response to the challenge using challenge ID `"22efa",response:"ae457b"`. The external challenge mechanism is enabled, NSO will invoke the configured executable and write `"[22efa;ae457b;]\n"` on the `stdin` stream for the executable. - -The task of the executable is then to validate the challenge ID, and response combination, thereby authenticating the user and also establishing the username and username-to-groups mapping. - -For example, the executable could be a RADIUS client which utilizes some proprietary vendor attributes to retrieve the username and groups of the user from the RADIUS server. If challenge ID, response validation is successful, the program should write `"accept "` followed by a space-separated list of groups the user is a member of, and additional information as described below. Again, assuming that `bob`'s challenge ID, the response combination indeed was `"22efa", "ae457b"` and that `bob` is a member of the `admin` and the `lamers` groups, the program should write `"accept admin lamers $uid $gid $supplementary_gids $HOME $USER\n"` on its standard output and then exit. - -{% hint style="info" %} -There is a general limit of 16000 bytes of output from the `externalchallenge` program. -{% endhint %} - -Thus the format of the output from an `externalchallenge` program when challenge-based authentication is successful should be: - -`"accept $groups $uid $gid $supplementary_gids $HOME $USER\n"` - -Where: - -* `$groups` is a space-separated list of the group names the user is a member of. -* `$uid` is the UNIX integer user ID NSO should use as a default when executing commands for this user. -* `$gid` is the UNIX integer group ID NSO should use as a default when executing commands for this user. -* `$supplementary_gids` is a (possibly empty) space-separated list of additional UNIX group IDs the user is also a member of. -* `$HOME` is the directory that should be used as HOME for this user when NSO executes commands on behalf of this user. -* `$USER` is the user derived from mapping the challenge ID, response. - -It is further possible for the program to return a token on successful authentication, by using `"accept_token"` instead of `"accept"`: - -`"accept_token $groups $uid $gid $supplementary_gids $HOME $USER $token\n"` - -Where: - -* `$token` is an arbitrary string. NSO will then, for some northbound interfaces, include this token in responses. - -It is also possible for the program to return additional information on successful authentication, by using `"accept_info"` instead of `"accept"`: - -`"accept_info $groups $uid $gid $supplementary_gids $HOME $USER $info\n"` - -Where: - -* `$info` is some arbitrary text. NSO will then just append this text to the generated audit log message (CONFD\_EXT\_LOGIN). - -Yet another possibility is for the program to return a warning that the user's password is about to expire, by using `"accept_warning"` instead of `"accept"`: - -`"accept_warning $groups $uid $gid $supplementary_gids $HOME $USER $warning\n"` - -Where: - -* `$warning` is an appropriate warning message. The message will be processed by NSO according to the setting of `/ncs-config/aaa/expiration-warning` in `ncs.conf`. - -There is also support for token variations of `"accept_info"` and `"accept_warning"` namely `"accept_token_info"` and `"accept_token_warning"`. Both `"accept_token_info"` and `"accept_token_warning"` expects the external program to output exactly the same as described above with the addition of a token after `$USER`: - -`"accept_token_info $groups $uid $gid $supplementary_gids $HOME $USER $token $info\n"` - -`"accept_token_warning $groups $uid $gid $supplementary_gids $HOME $USER $token $warning\n"` - -If authentication fails, the program should write `"reject"` or `"abort"`, possibly followed by a reason for the rejection and a trailing newline. For example `"reject Bad challenge response\n"` or just `"abort\n"`. The difference between `"reject"` and `"abort"` is that with `"reject"`, NSO will try subsequent mechanisms configured for `/ncs-config/aaa/challenge-order` in `ncs.conf` (if any), while with `"abort"`, the challenge-response authentication fails immediately. Thus `"abort"` can prevent subsequent mechanisms from being tried. Currently, the only available challenge-response authentication mechanism is the external one. - -Supported by some northbound APIs, such as JSON-RPC and CLI over SSH, the external challenge may also choose to issue a new challenge: - -`"challenge $challenge-id $challenge-prompt\n"` - -{% hint style="info" %} -The challenge prompt may be multi-line, so it must be base64 encoded. -{% endhint %} - -{% hint style="info" %} -Note that when using challenges with the CLI over SSH, the `/ncs-config/cli/ssh/use-keyboard-interactive>` need to be set to true for the challenges to be sent correctly to the client. -{% endhint %} - -{% hint style="info" %} -The configuration of the SSH client used may need to be given the option to allow a higher number of allowed number of password prompts, e.g. `-o NumberOfPasswordPrompts`, else the default number may introduce an unexpected behavior when the client is presented with multiple challenges. -{% endhint %} - -### Package Authentication - -The Package Authentication functionality allows for packages to handle the NSO authentication in a customized fashion. Authentication data can e.g. be stored remotely, and a script in the package is used to communicate with the remote system. - -Compared to external authentication, the Package Authentication mechanism allows specifying multiple packages to be invoked in the order they appear in the configuration. NSO provides implementations for LDAP, SAMLv2, and TACACS+ protocols with packages available in `$NCS_DIR/packages/auth/`. Additionally, you can implement your own authentication packages as detailed below. - -Authentication packages are NSO packages with the required content of an executable file `scripts/authenticate`. This executable basically follows the same API, and limitations, as the external auth script, but with a different input format and some additional functionality. Other than these requirements, it is possible to customize the package arbitrarily. - -{% hint style="info" %} -Package authentication is supported for Single Sign-On (see [Single Sign-On](../../development/advanced-development/web-ui-development/#single-sign-on-sso) in Web UI), JSON-RPC, and RESTCONF. Note that Single Sign-On and (non-batch) JSON-RPC allow all functionality while the RESTCONF interface will treat anything other than a "`accept_username`" reply from the package as if authentication failed! -{% endhint %} - -Package authentication is enabled by setting the `ncs.conf` options `/ncs-config/aaa/package-authentication/enabled` to true, and adding the package by name in the `/ncs-config/aaa/package-authentication/packages` list. The order of the configured packages is the order that the packages will be used when attempting to authenticate a user. See [ncs.conf(5)](../../resources/man/ncs.conf.5.md) in Manual Pages for details. - -If this feature is configured in `ncs.conf`, NSO will for each configured package invoke `script/authenticate`, and pass username, password, and original HTTP request (i.e. the user-supplied `next` query parameter), HTTP request, HTTP headers, HTTP body, client source IP, client source port, northbound API context, and protocol on `stdin` using the string notation: `"[user;password;orig_request;request;headers;body;src-ip;src-port;ctx;proto;]\n"`. - -{% hint style="info" %} -The fields user, password, orig\_request, request, headers, and body are all base64 encoded. -{% endhint %} - -{% hint style="info" %} -If the body length exceeds the `partial_post_size` of the RESTCONF server, the body passed to the authenticate script will only contain the string `'==nso_package_authentication_partial_body==`'. -{% endhint %} - -{% hint style="info" %} -The original request will be prefixed with the string `==nso_package_authentication_next==` before the base64 encoded part. This means supplying the `next` query parameter value `/my-location` will pass the following string to the authentication script: `==nso_package_authentication_next==L215LWxvY2F0aW9u`. -{% endhint %} - -For example, if an unauthenticated user attempts to start a single sign-on process over northbound HTTP-based APIs with the cisco-nso-saml2-auth package, package authentication is enabled and configured with packages, and also single sign-on is enabled, NSO will, for each configured package, invoke the executable `scripts/authenticate` and write `"[;;;R0VUIC9zc28vc2FtbC9sb2dpbi8gSFRUUC8xLjE=;;;127.0.0.1;59226;webui;https;]\n"`. on the `stdin` stream for the executable. - -For clarity, the base64 decoded contents sent to `stdin` presented: `"[;;;GET /sso/saml/login/ HTTP/1.1;;;127.0.0.1;54321;webui;https;]\n"`. - -The task of the package is then to authenticate the user and also establish the username-to-groups mapping. - -For example, the package could support a SAMLv2 authentication protocol which communicates with an Identity Provider (IdP) for authentication. If authentication is successful, the program should write either `"accept"`, or `"accept_username"`, depending on whether the authentication is started with a username or if an external entity handles the entire authentication and supplies the username for a successful authentication. (SAMLv2 uses `accept_username`, since the IdP handles the entire authentication.) The "accept\_username " is followed by a username and then followed by a space-separated list of groups the user is a member of, and additional information as described below. If authentication is successful and the authenticated user `bob` is a member of the groups `admin` and `wheel`, the program should write `"accept_username bob admin wheel 1000 1000 100 /home/bob\n"` on its standard output and then exit. - -{% hint style="info" %} -There is a general limit of 16000 bytes of output from the "packageauth" program. -{% endhint %} - -Thus the format of the output from a `packageauth` program when authentication is successful should be either the same as from `externalauth` (see [External Authentication](aaa-infrastructure.md#ug.aaa.external_authentication)) or the following: - -`"accept_username $USER $groups $uid $gid $supplementary_gids $HOME\n"` - -Where: - -* `$USER` is the user derived during the execution of the "packageauth" program. -* `$groups` is a space-separated list of the group names the user is a member of. -* `$uid` is the UNIX integer user ID NSO should use as a default when executing commands for this user. -* `$gid` is the UNIX integer group ID NSO should use as a default when executing commands for this user. -* `$supplementary_gids` is a (possibly empty) space-separated list of additional UNIX group IDs the user is also a member of. -* `$HOME` is the directory that should be used as HOME for this user when NSO executes commands on behalf of this user. - -In addition to the `externalauth` API, the authentication packages can also return the following responses: - -* `unknown '`_`reason`_`'` - (_`reason`_ being plain-text) if they can't handle authentication for the supplied input. -* `redirect '`_`url`_`'` - (_`url`_ being base64 encoded) for an HTTP redirect. -* `content '`_`content-type`_`' '`_`content`_`'` - (_`content-type`_ being plain-text mime-type and _`content`_ being base64 encoded) to relay supplied content. -* `accept_username_redirect url $USER $groups $uid $gid $supplementary_gids $HOME` - which combines the `accept_username` and `redirect`. - -It is also possible for the program to return additional information on successful authentication, by using `"accept_info"` instead of `"accept"`: - -`"accept_info $groups $uid $gid $supplementary_gids $HOME $info\n"` - -Where: - -* `$info` is some arbitrary text. NSO will then just append this text to the generated audit log message (NCS\_PACKAGE\_AUTH\_SUCCESS). - -Yet another possibility is for the program to return a warning that the user's password is about to expire, by using `"accept_warning"` instead of `"accept"`: - -`"accept_warning $groups $uid $gid $supplementary_gids $HOME $warning\n"` - -Where: - -* `$warning` is an appropriate warning message. The message will be processed by NSO according to the setting of `/ncs-config/aaa/expiration-warning` in `ncs.conf`. - -If authentication fails, the program should write `"reject"` or `"abort"`, possibly followed by a reason for the rejection and a trailing newline. For example `"reject 'Bad password'\n"` or just `"abort\n"`. The difference between `"reject"` and `"abort"` is that with `"reject"`, NSO will try subsequent mechanisms configured for `/ncs-config/aaa/auth-order`, and packages configured for `/ncs-config/aaa/package-authentication/packages` in `ncs.conf` (if any), while with `"abort"`, the authentication fails immediately. Thus `"abort"` can prevent subsequent mechanisms from being tried, but when external authentication is the last mechanism (as in the default order), it has the same effect as `"reject"`. - -When package authentication is used, the group list returned by the package executable is prepended by any possible group information stored locally under the `/aaa` tree. Hence when package authentication is used, it is indeed possible to have the entire `/aaa/authentication` tree empty. The group assignment performed by the external program will still be valid and the relevant groups will be used by NSO when the authorization rules are checked. - -### **Username/Password Package Authentication for CLI** - -Package authentication will invoke the `scripts/authenticate` when a user tries to authenticate using CLI. In this case, only the username, password, client source IP, client source port, northbound API context, and protocol will be passed to the script. - -{% hint style="info" %} -When serving a username/password request, script output other than accept, challenge or abort will be treated as if authentication failed. -{% endhint %} - -### **Package Challenges** - -When this is enabled, `/ncs-config/aaa/package-authentication/package-challenge/enabled` is set to true, packages will also be used to try to resolve challenges sent to the server and are only supported by CLI over SSH. The script `script/challenge` will be invoked passing challenge ID, response, client source IP, client source port, northbound API context, and protocol on `stdin` using the string notation: `"[challengeid;response;src-ip;src-port;ctx;proto;]\n"` . The output should follow that of the authenticate script. - -{% hint style="info" %} -The fields `challengeid` and response are base64 encoded when passed to the script. -{% endhint %} - -## Authenticating IPC Access - -NSO communicates with clients (Python and Java client libraries, `ncs_cli`, `netconf-subsys`, and others) using the NSO IPC socket. The protocol used allows the client to provide user and group information to use for authorization in NSO, effectively delegating authentication to the client. - -By default, only local connections to the IPC socket are allowed. If all local clients are considered trusted, the socket can provide unauthenticated access, with the client-supplied user name. This is what the `--user` option of `ncs_cli` does. For example, the following connects to NSO as user `admin`. - -```bash -ncs_cli --user admin -``` - -The same is possible for the group. This unauthenticated access is currently the default. - -The main condition here is that all clients connecting to the socket are trusted to use the correct user and group information. That is often not the case, such as untrusted users having shell access to the host to run `ncs_cli` or otherwise initiate local connections to the IPC socket. Then access to the socket must be restricted. - -In general, authenticating access to the IPC socket is a security best practice and should always be used. When NSO is configured to use Unix domain sockets for IPC, it authenticates the client based on the UID of the other end of the socket connection. Alternatively, the system can be instructed to use TCP sockets. In this case, the system should be configured to use an access check, where every IPC client must prove that it has access to a pre-shared key. See [Restricting Access to the IPC Socket](../advanced-topics/ipc-connection.md#restricting-access-to-the-ipc-socket) on how to enable it. - -### UID-based Authentication for Unix Sockets - -NSO will use Unix domain sockets for IPC communications when `ncs-local-ipc/enabled` configuration in `ncs.conf` is set to true. The main benefit of this communication method is that it is generally more secure than TCP sockets. It also provides additional information on the communicating peer, such as the user ID of the calling process. NSO can then use this information to authenticate the peer. - -As part of the initial handshake, NSO reads the effective UID (euid) of the process initiating the Unix socket connection. The system then finds an `/aaa/authentication/users/user` entry with the corresponding `uid` value. Access is permitted or denied based on the `local_ipc_access` value. If access is permitted, the user connects as the user, found in the `/aaa/authentication/users/user` list. The following is an example of such a user list entry: - -```bash -aaa authentication users user admin - uid 500 - gid 500 - password $6$... - ssh_keydir /var/ncs/homes/admin/.ssh - homedir /var/ncs/homes/admin - local_ipc_access true -! -``` - -NSO will skip this access check in case the euid of the connecting process is 0 (root user) or same as the user NSO is running as. (In both these cases, the connecting user could access NSO data directly, bypassing the access check.) - -If using Unix socket IPC, clients and client libraries must now specify the path that identifies the socket. The path must match the one set under `ncs-local-ipc/path` in `ncs.conf`. Clients may expose a client-specific way to set it, such as the `-S` option of the `ncs_cli` command. Alternatively, you can use the `NCS_IPC_PATH` environment variable to specify the socket path independently of the used client. - -See [examples.ncs/aaa/ipc](https://github.com/NSO-developer/nso-examples/tree/6.6/aaa/ipc) for a working example. - -## Group Membership - -Once a user is authenticated, group membership must be established. A single user can be a member of several groups. Group membership is used by the authorization rules to decide which operations a certain user is allowed to perform. Thus, the NSO AAA authorization model is entirely group-based. This is also sometimes referred to as role-based authorization. - -All groups are stored under `/nacm/groups`, and each group contains a number of usernames. The `ietf-netconf-acm.yang` model defines a group entry: - -```yang -list group { - key name; - - description - "One NACM Group Entry. This list will only contain - configured entries, not any entries learned from - any transport protocols."; - - leaf name { - type group-name-type; - description - "Group name associated with this entry."; - } - - leaf-list user-name { - type user-name-type; - description - "Each entry identifies the username of - a member of the group associated with - this entry."; - } -} -``` - -The `tailf-acm.yang` model augments this with a `gid` leaf: - -```yang -augment /nacm:nacm/nacm:groups/nacm:group { - leaf gid { - type int32; - description - "This leaf associates a numerical group ID with the group. - When a OS command is executed on behalf of a user, - supplementary group IDs are assigned based on 'gid' values - for the groups that the use is a member of."; - } -} -``` - -A valid group entry could thus look like: - -```xml - - admin - bob - joe - 99 - -``` - -The above XML data would then mean that users `bob` and `joe` are members of the `admin` group. The users need not necessarily exist as actual users under `/aaa/authentication/users` in order to belong to a group. If for example PAM authentication is used, it does not make sense to have all users listed under `/aaa/authentication/users`. - -By default, the user is assigned to groups by using any groups provided by the northbound transport (e.g. via the `ncs_cli` or `netconf-subsys` programs), by consulting data under `/nacm/groups`, by consulting the `/etc/group` file, and by using any additional groups supplied by the authentication method. If `/nacm/enable-external-groups` is set to "false", only the data under `/nacm/groups` is consulted. - -The resulting group assignment is the union of these methods, if it is non-empty. Otherwise, the default group is used, if configured ( `/ncs-config/aaa/default-group` in `ncs.conf`). - -A user entry has a UNIX uid and UNIX gid assigned to it. Groups may have optional group IDs. When a user is logged in, and NSO tries to execute commands on behalf of that user, the uid/gid for the command execution is taken from the user entry. Furthermore, UNIX supplementary group IDs are assigned according to the `gid`'s in the groups where the user is a member. - -## Authorization - -Once a user is authenticated and group membership is established, when the user starts to perform various actions, each action must be authorized. Normally the authorization is done based on rules configured in the AAA data model as described in this section. - -The authorization procedure first checks the value of `/nacm/enable-nacm`. This leaf has a default of `true`, but if it is set to `false`, all access is permitted. Otherwise, the next step is to traverse the `rule-list` list: - -```yang -list rule-list { - key "name"; - ordered-by user; - description - "An ordered collection of access control rules."; - - leaf name { - type string { - length "1..max"; - } - description - "Arbitrary name assigned to the rule-list."; - } - leaf-list group { - type union { - type matchall-string-type; - type group-name-type; - } - description - "List of administrative groups that will be - assigned the associated access rights - defined by the 'rule' list. - - The string '*' indicates that all groups apply to the - entry."; - } - - // ... -} -``` - -If the `group` leaf-list in a `rule-list` entry matches any of the user's groups, the `cmdrule` list entries are examined for command authorization, while the `rule` entries are examined for RPC, notification, and data authorization. - -### Command Authorization - -The `tailf-acm.yang` module augments the `rule-list` entry in `ietf-netconf-acm.yang` with a `cmdrule` list: - -```yang -augment /nacm:nacm/nacm:rule-list { - - list cmdrule { - key "name"; - ordered-by user; - description - "One command access control rule. Command rules control access - to CLI commands and Web UI functions. - - Rules are processed in user-defined order until a match is - found. A rule matches if 'context', 'command', and - 'access-operations' match the request. If a rule - matches, the 'action' leaf determines if access is granted - or not."; - - leaf name { - type string { - length "1..max"; - } - description - "Arbitrary name assigned to the rule."; - } - - leaf context { - type union { - type nacm:matchall-string-type; - type string; - } - default "*"; - description - "This leaf matches if it has the value '*' or if its value - identifies the agent that is requesting access, i.e. 'cli' - for CLI or 'webui' for Web UI."; - } - - leaf command { - type string; - default "*"; - description - "Space-separated tokens representing the command. Refer - to the Tail-f AAA documentation for further details."; - } - - leaf access-operations { - type union { - type nacm:matchall-string-type; - type nacm:access-operations-type; - } - default "*"; - description - "Access operations associated with this rule. - - This leaf matches if it has the value '*' or if the - bit corresponding to the requested operation is set."; - } - - leaf action { - type nacm:action-type; - mandatory true; - description - "The access control action associated with the - rule. If a rule is determined to match a - particular request, then this object is used - to determine whether to permit or deny the - request."; - } - - leaf log-if-permit { - type empty; - description - "If this leaf is present, access granted due to this rule - is logged in the developer log. Otherwise, only denied - access is logged. Mainly intended for debugging of rules."; - } - - leaf comment { - type string; - description - "A textual description of the access rule."; - } - } -} -``` - -Each rule has seven leafs. The first is the `name` list key, the following three leafs are matching leafs. When NSO tries to run a command, it tries to match the command towards the matching leafs and if all of `context`, `command`, and `access-operations` match, the fifth field, i.e. the `action`, is applied. - -* `name`: `name` is the name of the rule. The rules are checked in order, with the ordering given by the YANG `ordered-by user` semantics, i.e. independent of the key values. -* `context`: `context` is either of the strings `cli`, `webui`, or `*` for a command rule. This means that we can differentiate authorization rules for which access method is used. Thus if command access is attempted through the CLI, the context will be the string `cli` whereas for operations via the Web UI, the context will be the string `webui`. -* `command`: This is the actual command getting executed. If the rule applies to one or several CLI commands, the string is a space-separated list of CLI command tokens, for example `request system reboot`. If the command applies to Web UI operations, it is a space-separated string similar to a CLI string. A string that consists of just `*` matches any command.\ - \ - In general, we do not recommend using command rules to protect the configuration. Use rules for data access as described in the next section to control access to different parts of the data. Command rules should be used only for CLI commands and Web UI operations that cannot be expressed as data rules.\ - \ - The individual tokens can be POSIX extended regular expressions. Each regular expression is implicitly anchored, i.e. an `^` is prepended and a `$` is appended to the regular expression. -* `access-operations`: `access-operations` is used to match the operation that NSO tries to perform. It must be one or both of the "read" and "exec" values from the `access-operations-type` bits type definition in `ietf-netconf-acm.yang`, or "\*" to match any operation. -* action: If all of the previous fields match, the rule as a whole matches and the value of `action` will be taken. I.e. if a match is found, a decision is made whether to permit or deny the request in its entirety. If `action` is `permit`, the request is permitted, if `action` is `deny`, the request is denied and an entry is written to the developer log. -* `log-if-permit`: If this leaf is present, an entry is written to the developer log for a matching request also when `action` is `permit`. This is very useful when debugging command rules. -* `comment`: An optional textual description of the rule. - -For the rule processing to be written to the devel log, the `/ncs-config/logs/developer-log-level` entry in `ncs.conf` must be set to `trace`. - -If no matching rule is found in any of the `cmdrule` lists in any `rule-list` entry that matches the user's groups, this augmentation from `tailf-acm.yang` is relevant: - -```yang -augment /nacm:nacm { - leaf cmd-read-default { - type nacm:action-type; - default "permit"; - description - "Controls whether command read access is granted - if no appropriate cmdrule is found for a - particular command read request."; - } - - leaf cmd-exec-default { - type nacm:action-type; - default "permit"; - description - "Controls whether command exec access is granted - if no appropriate cmdrule is found for a - particular command exec request."; - } - - leaf log-if-default-permit { - type empty; - description - "If this leaf is present, access granted due to one of - /nacm/read-default, /nacm/write-default, or /nacm/exec-default - /nacm/cmd-read-default, or /nacm/cmd-exec-default - being set to 'permit' is logged in the developer log. - Otherwise, only denied access is logged. Mainly intended - for debugging of rules."; - } -} -``` - -* If `read` access is requested, the value of `/nacm/cmd-read-default` determines whether access is permitted or denied. -* If `exec` access is requested, the value of `/nacm/cmd-exec-default` determines whether access is permitted or denied. - -If `access` is permitted due to one of these default leafs, the `/nacm/log-if-default-permit`has the same effect as the `log-if-permit` leaf for the `cmdrule` lists. - -### RPC, Notification, and Data Authorization - -The rules in the `rule` list are used to control access to rpc operations, notifications, and data nodes defined in YANG models. Access to invocation of actions (`tailf:action`) is controlled with the same method as access to data nodes, with a request for `exec` access. `ietf-netconf-acm.yang` defines a `rule` entry as: - -```yang -list rule { - key "name"; - ordered-by user; - description - "One access control rule. - - Rules are processed in user-defined order until a match is - found. A rule matches if 'module-name', 'rule-type', and - 'access-operations' match the request. If a rule - matches, the 'action' leaf determines if access is granted - or not."; - - leaf name { - type string { - length "1..max"; - } - description - "Arbitrary name assigned to the rule."; - } - - leaf module-name { - type union { - type matchall-string-type; - type string; - } - default "*"; - description - "Name of the module associated with this rule. - - This leaf matches if it has the value '*' or if the - object being accessed is defined in the module with the - specified module name."; - } - choice rule-type { - description - "This choice matches if all leafs present in the rule - match the request. If no leafs are present, the - choice matches all requests."; - case protocol-operation { - leaf rpc-name { - type union { - type matchall-string-type; - type string; - } - description - "This leaf matches if it has the value '*' or if - its value equals the requested protocol operation - name."; - } - } - case notification { - leaf notification-name { - type union { - type matchall-string-type; - type string; - } - description - "This leaf matches if it has the value '*' or if its - value equals the requested notification name."; - } - } - case data-node { - leaf path { - type node-instance-identifier; - mandatory true; - description - "Data Node Instance Identifier associated with the - data node controlled by this rule. - - Configuration data or state data instance - identifiers start with a top-level data node. A - complete instance identifier is required for this - type of path value. - - The special value '/' refers to all possible - data-store contents."; - } - } - } - - leaf access-operations { - type union { - type matchall-string-type; - type access-operations-type; - } - default "*"; - description - "Access operations associated with this rule. - - This leaf matches if it has the value '*' or if the - bit corresponding to the requested operation is set."; - } - - leaf action { - type action-type; - mandatory true; - description - "The access control action associated with the - rule. If a rule is determined to match a - particular request, then this object is used - to determine whether to permit or deny the - request."; - } - - leaf comment { - type string; - description - "A textual description of the access rule."; - } -} -``` - -`tailf-acm` augments this with two additional leafs: - -```yang -augment /nacm:nacm/nacm:rule-list/nacm:rule { - - leaf context { - type union { - type nacm:matchall-string-type; - type string; - } - default "*"; - description - "This leaf matches if it has the value '*' or if its value - identifies the agent that is requesting access, e.g. 'netconf' - for NETCONF, 'cli' for CLI, or 'webui' for Web UI."; - - } - - leaf log-if-permit { - type empty; - description - "If this leaf is present, access granted due to this rule - is logged in the developer log. Otherwise, only denied - access is logged. Mainly intended for debugging of rules."; - } -} -``` - -Similar to the command access check, whenever a user through some agent tries to access an RPC, a notification, a data item, or an action, access is checked. For a rule to match, three or four leafs must match and when a match is found, the corresponding action is taken. - -We have the following leafs in the `rule` list entry. - -* `name`: The name of the rule. The rules are checked in order, with the ordering given by the YANG `ordered-by user` semantics, i.e., independent of the key values. -* `module-name`: The `module-name` string is the name of the YANG module where the node being accessed is defined. The special value `*` (i.e., the default) matches all modules.\ - **Note**: Since the elements of the path to a given node may be defined in different YANG modules when augmentation is used, rules that have a value other than `*` for the `module-name` leaf may require that additional processing is done before a decision to permit or deny, or the access can be taken. Thus, if an XPath that completely identifies the nodes that the rule should apply to is given for the `path` leaf (see below), it may be best to leave the `module-name` leaf unset. -* `rpc-name / notification-name / path`: This is a choice between three possible leafs that are used for matching, in addition to the `module-name`: -* `rpc-name`: The name of an RPC operation, or `*` to match any RPC. -* `notification-name`: the name of a notification, or `*` to match any notification. -* `path`: A restricted XPath expression leading down into the populated XML tree. A rule with a path specified matches if it is equal to or shorter than the checked path. Several types of paths are allowed. - - 1. Tagpaths that do not contain any keys. For example `/ncs/live-device/live-status`. - 2. Instantiated key: as in `/devices/device[name="x1"]/config/interface` matches the interface configuration for managed device "x1" It's possible to have partially instantiated paths only containing some keys instantiated - i.e. combinations of tagpaths and keypaths. Assuming a deeper tree, the path `/devices/device/config/interface[name="eth0"]` matches the `eth0` interface configuration on all managed devices. - 3. The wild card at the end as in: `/services/web-site/*` does not match the website service instances, but rather all children of the website service instances. - 4. The leading/trailing whitespace as in: `" /devices/device/config "` are ignored. - - Thus, the path in a rule is matched against the path in the attempted data access. If the attempted access has a path that is equal to or longer than the rule path - we have a match.\ - \ - If none of the leafs `rpc-name`, `notification-name`, or `path` are set, the rule matches for any RPC, notification, data, or action access. -* `context`: `context` is either of the strings `cli`, `netconf`, `webui`, `snmp`, or `*` for a data rule. Furthermore, when we initiate user sessions from MAAPI, we can choose any string we want. Similarly to command rules, we can differentiate access depending on which agent is used to gain access. -* `access-operations`: `access-operations` is used to match the operation that NSO tries to perform. It must be one or more of the "create", "read", "update", "delete" and "exec" values from the `access-operations-type` bits type definition in `ietf-netconf-acm.yang`, or "\*" to match any operation. -* `action`: This leaf has the same characteristics as the `action` leaf for command access. -* `log-if-permit`: This leaf has the same characteristics as the `log-if-permit` leaf for command access. -* `comment`: An optional textual description of the rule. - -If no matching rule is found in any of the `rule` lists in any `rule-list` entry that matches the user's groups, the data model node for which access is requested is examined for the presence of the NACM extensions: - -* If the `nacm:default-deny-all` extension is specified for the data model node, the access is denied. -* If the `nacm:default-deny-write` extension is specified for the data model node, and `create`, `update`, or `delete` access is requested, the access is denied. - -If examination of the NACM extensions did not result in access being denied, the value (`permit` or `deny`) of the relevant default leaf is examined: - -* If `read` access is requested, the value of `/nacm/read-default` determines whether access is permitted or denied. -* If `create`, `update`, or `delete` access is requested, the value of `/nacm/write-default` determines whether access is permitted or denied. -* If `exec` access is requested, the value of `/nacm/exec-default` determines whether access is permitted or denied. - -If access is permitted due to one of these default leafs, this augmentation from `tailf-acm.yang` is relevant: - -```yang -augment /nacm:nacm { - ... - leaf log-if-default-permit { - type empty; - description - "If this leaf is present, access granted due to one of - /nacm/read-default, /nacm/write-default, /nacm/exec-default - /nacm/cmd-read-default, or /nacm/cmd-exec-default - being set to 'permit' is logged in the developer log. - Otherwise, only denied access is logged. Mainly intended - for debugging of rules."; - } -} -``` - -I.e., it has the same effect as the `log-if-permit` leaf for the `rule` lists, but for the case where the value of one of the default leafs permits access. - -When NSO executes a command, the command rules in the authorization database are searched, The rules are tried in order, as described above. When a rule matches the operation (command) that NSO is attempting, the action of the matching rule is applied — whether permit or deny. - -When actual data access is attempted, the data rules are searched. E.g., when a user attempts to execute `delete aaa` in the CLI, the user needs delete access to the entire tree `/aaa`. - -Another example is if a CLI user writes `show configuration aaa` TAB, it suffices to have read access to at least one item below `/aaa` for the CLI to perform the TAB completion. If no rule matches or an explicit deny rule is found, the CLI will not TAB-complete. - -Yet another example is if a user tries to execute `delete aaa authentication users`, we need to perform a check on the paths `/aaa` and `/aaa/authentication` before attempting to delete the sub-tree. Say that we have a rule for path `/aaa/authentication/users` which is a permit rule and we have a subsequent rule for path `/aaa` which is a deny rule. With this rule set the user should indeed be allowed to delete the entire `/aaa/authentication/users` tree but not the `/aaa` tree nor the `/aaa/authentication` tree. - -We have two variations on how the rules are processed. The easy case is when we actually try to read or write an item in the configuration database. The execution goes like this: - -``` -foreach rule { - if (match(rule, path)) { - return rule.action; - } -} -``` - -The second case is when we execute TAB completion in the CLI. This is more complicated. The execution goes like this: - -``` -rules = select_rules_that_may_match(rules, path); -if (any_rule_is_permit(rules)) - return permit; -else - return deny; -``` - -The idea is that as we traverse (through TAB) down the XML tree, as long as there is at least one rule that can possibly match later, once we have more data, we must continue. For example, assume we have: - -1. `"/system/config/foo" --> permit` -2. `"/system/config" --> deny` - -If we in the CLI stand at `"/system/config"` and hit TAB we want the CLI to show `foo` as a completion, but none of the other nodes that exist under `/system/config`. Whereas if we try to execute `delete /system/config` the request must be rejected. - -By default, NACM rules are configured for the entire `tailf:action` or YANG 1.1 `action` statements, but not for `input` statement child leafs. To override this behavior, and enable NACM rules on `input` leafs, set the following parameter to 'true': `/ncs-config/aaa/action-input-rules/enabled`. When enabled all action input leafs given to an action will be validated for NACM rules. If broad 'deny' NACM rules are used, you might need to add 'permit' rules for the affected action input leafs to allow actions to be used with parameters. - -### NACM Rules and Services - -By design NACM rules are ignored for changes done by services — FASTMAP, Reactive FASTMAP, or Nano services. The reasoning behind this is that a service package can be seen as a controlled way to provide limited access to devices for a user group that is not allowed to apply arbitrary changes on the devices. - -However, there are NSO installations where this behavior is not desired, and NSO administrators want to enforce NACM rules even on changes done by services. For this purpose, the leaf called `/nacm/enforce-nacm-on-services` is provided. By default, it is set to `false`. - -Note however that currently, even with this leaf set to true, there are limitations. Namely, the post-actions for nano-services are run in a user session without any access checks. Besides that, NACM rules are not enforced on the read operations performed in the service callbacks. - -It might be desirable to deny everything for a user group and only allow access to a specific service. This pattern could be used to allow an operator to provision the service, but deny everything else. While this pattern works for a normal FASTMAP service, there are some caveats for stacked services, Reactive FASTMAP, and Nano services. For these kinds of services, in addition to the service itself, access should be provided to the user group for the following paths: - -* In case of stacked services, the user group needs read and write access to the leaf `private/re-deploy-counter` under the bottom service. Otherwise, the user will not be able to redeploy the service. -* In the case of Reactive FASTMAP or Nano services, the user group needs read and write access to the following: - * `/zombies` - * `/side-effect-queue` - * `/kickers` - -### Device Group Authorization - -In deployments with many devices, it can become cumbersome to handle data authorization per device. To help with this there is a rule type that works on device group membership (for more on device groups, see [Device Groups](../../operation-and-usage/operations/nso-device-manager.md#user_guide.devicemanager.device_groups)). To do this, devices are added to different device groups, and the rule type `device-group-rule` is used. - -The IETF NACM rule type is augmented with a new rule type named `device-group-rule` which contains a leafref to the device groups. See the following example. - -{% code title="Device Group Model Augmentation" %} -```yang -augment "/nacm:nacm/nacm:rule-list/nacm:rule/nacm:rule-type" { - case device-group-rule { - leaf device-group { - type leafref { - path "/ncs:devices/ncs:device-group/ncs:name"; - } - description - "Which device group this rule applies to."; - } - } -} -``` -{% endcode %} - -In the example below, we configure two device groups based on different regions and add devices to them. - -{% code title="Device Group Configuration" %} -```xml - - - us_east - cli0 - gen0 - - - us_west - nc0 - - -``` -{% endcode %} - -In the example below, we configure an operator for the `us_east` region: - -{% code title="NACM Group Configuration" %} -```xml - - - - us_east - us_east_oper - - - -``` -{% endcode %} - -\ -In the example below, we configure the device group rules and refer to the device group and the `us_east` group. - -{% code title="Device Group Authorization Rules" %} -```xml - - - us_east - us_east - - us_east_read_permit - us_east - read - permit - - - us_east_create_permit - us_east - create - permit - - - us_east_update_permit - us_east - update - permit - - - us_east_delete_permit - us_east - delete - permit - - - -``` -{% endcode %} - -In summary device group authorization gives a more compact configuration for deployments where devices can be grouped and authorization can be done on a device group basis. - -Modifications on the device-group subtree are recommended to be controlled by a limited set of users. - -### Authorization Examples - -Assume that we have two groups, `admin` and `oper`. We want `admin` to be able to see and edit the XML tree rooted at `/aaa`, but we do not want users who are members of the `oper` group to even see the `/aaa` tree. We would have the following rule list and rule entries. Note, here we use the XML data from `tailf-aaa.yang` to exemplify. The examples apply to all data, for all data models loaded into the system. - -```xml - - admin - admin - - tailf-aaa - tailf-aaa - / - read create update delete - permit - - - - oper - oper - - tailf-aaa - tailf-aaa - / - read create update delete - deny - - -``` - -If we do not want the members of `oper` to be able to execute the NETCONF operation `edit-config`, we define the following rule list and rule entries: - -```xml - - oper - oper - - edit-config - edit-config - netconf - exec - deny - - -``` - -To spell it out, the above defines four elements to match. If NSO tries to perform a `netconf` operation, which is the operation `edit-config`, and the user who runs the command is a member of the `oper` group, and finally it is an `exec` (execute) operation, we have a match. If so, the action is `deny`. - -The `path` leaf can be used to specify explicit paths into the XML tree using XPath syntax. For example the following: - -```xml - - admin - admin - - bob-password - /aaa/authentication/users/user[name='bob']/password - cli - read update - permit - - -``` - -Explicitly allows the `admin` group to change the password for precisely the `bob` user when the user is using the CLI. Had `path` been `/aaa/authentication/users/user/password` the rule would apply to all password elements for all users. Since the `path` leaf completely identifies the nodes that the rule applies to, we do not need to give `tailf-aaa` for the `module-name` leaf. - -NSO applies variable substitution, whereby the username of the logged-in user can be used in a `path`. Thus: - -```xml - - admin - admin - - user-password - /aaa/authentication/users/user[name='$USER']/password - cli - read update - permit - - -``` - -The above rule allows all users that are part of the `admin` group to change their own passwords only. - -A member of `oper` is able to execute NETCONF operation `action` if that member has `exec` access on NETCONF RPC `action` operation, `read` access on all instances in the hierarchy of data nodes that identifies the specific action in the data store, and `exec` access on the specific action. For example, an action is defined as below. - -```yang -container test { - action double { - input { - leaf number { - type uint32; - } - } - output { - leaf result { - type uint32; - } - } - } -} -``` - -To be able to execute `double` action through NETCONF RPC, the members of `oper` need the following rule list and rule entries. - -```xml - - oper - oper - - - allow-netconf-rpc-action - action - netconf - exec - permit - - - allow-read-test - /test - read - permit - - - allow-exec-double - /test/double - exec - permit - - -``` - -Or, a simpler rule set as the following. - -```xml - - oper - oper - - - allow-netconf-rpc-action - action - netconf - exec - permit - - - allow-exec-double - /test - read exec - permit - - -``` - -Finally, if we wish members of the `oper` group to never be able to execute the `request system reboot` command, also available as a `reboot` NETCONF rpc, we have: - -```xml - - oper - oper - - - request-system-reboot - cli - request system reboot - exec - deny - - - - - - - request-reboot - cli - request reboot - exec - deny - - - - netconf-reboot - reboot - netconf - exec - deny - - - -``` - -### Troubleshooting NACM Rules - -In this section, we list some tips to make it easier to troubleshoot NACM rules. - -{% hint style="success" %} -Use `log-if-permit` and `log-if-default-permit` together with the developer log level set to `trace`. -{% endhint %} - -Use the `tailf-acm.yang` module augmentation `log-if-permit` leaf for rules with `action` `permit`. When those rules trigger a permit action a trace entry is added to the developer log. To see trace entries make sure the `/ncs-config/logs/developer-log-level` is set to `trace`. - -If you have a default rule with `action` `permit` you can use the `log-if-default-permit` leaf instead. - -{% hint style="success" %} -NACM rules are read at the start of the session and are used throughout the session. -{% endhint %} - -When a user session is created it will gather the authorization rules that are relevant for that user's group(s). The rules are used throughout the user session lifetime. When we update the AAA rules the active sessions are not affected. For example, if an administrator updates the NACM rules in one session the update will not apply to any other currently active sessions. The updates will apply to new sessions created after the update. - -{% hint style="success" %} -Explicitly state NACM groups when starting the CLI. For example `ncs_cli -u oper -g oper`. -{% endhint %} - -It is the user's group membership that determines what rules apply. Starting the CLI using the `ncs_cli` command without explicitly setting the groups, defaults to the actual UNIX groups the user is a member of. On Darwin, one of the default groups is usually `admin`, which can lead to the wrong group being used. - -{% hint style="success" %} -Be careful with namespaces in rulepaths. -{% endhint %} - -Unless a rulepath is made explicit by specifying namespace it will apply to that specific path in all namespaces. Below we show parts of an example from [RFC 8341](https://tools.ietf.org/html/rfc8341), where the `path` element has an `xmlns` attribute and the path is namespaced. If these would not have been namespaced, the rules would not behave as expected. - -{% code title="Example: Excerpt from RFC 8341 Appendix A.4" %} -```xml - - permit-acme-config - - /acme:acme-netconf/acme:config-parameters - - ... -``` -{% endcode %} - -\ -In the example above (Excerpt from RFC 8341 Appendix A.4), the path is namespaced. - -## The AAA Cache - -NSO's AAA subsystem will cache the AAA information in order to speed up the authorization process. This cache must be updated whenever there is a change to the AAA information. The mechanism for this update depends on how the AAA information is stored, as described in the following two sections. - -### Populating AAA using CDB - -To start NSO, the data models for AAA must be loaded. The defaults in the case that no actual data is loaded for these models allow all read and exec access, while write access is denied. Access may still be further restricted by the NACM extensions, though — e.g., the `/nacm` container has `nacm:default-deny-all`, meaning that not even read access is allowed if no data is loaded. - -The NSO installation ships with an XML initialization file containing AAA configuration. The file is called `aaa_init.xml` and is, by default, copied to the CDB directory by the NSO install scripts. - -The local installation variant, targeting development only, defines two users, `admin` and `oper` with passwords set to `admin` and `oper` respectively for authentication. The two users belong to user groups with NACM rules restricting their authorization level. The system installation `aaa_init.xml` variant, targeting production deployment, defines NACM rules only as users are, by default, authenticated using PAM. The NACM rules target two user groups, `ncsadmin` and `ncsoper`. Users belonging to the `ncsoper` group are limited to read-only access. - -{% hint style="info" %} -The default `aaa_init.xml` file provided with the NSO system installation must not be used as-is in a deployment without reviewing and verifying that every NACM rule in the file matches -{% endhint %} - -Normally the AAA data will be stored as configuration in CDB. This allows for changes to be made through NSO's transaction-based configuration management. In this case, the AAA cache will be updated automatically when changes are made to the AAA data. If changing the AAA data via NSO's configuration management is not possible or desirable, it is alternatively possible to use the CDB operational data store for AAA data. In this case, the AAA cache can be updated either explicitly e.g. by using the `maapi_aaa_reload()` function, see the [confd\_lib\_maapi(3)](../../resources/man/confd_lib_maapi.3.md) in the Manual Pages manual page, or by triggering a subscription notification by using the subscription lock when updating the CDB operational data store, see [Using CDB](../../development/core-concepts/using-cdb.md) in Development. - -### Hiding the AAA Tree - -Some applications may not want to expose the AAA data to end users in the CLI or the Web UI. Two reasonable approaches exist here and both rely on the `tailf:export` statement. If a module has `tailf:export none` it will be invisible to all agents. We can then either use a transform whereby we define another AAA model and write a transform program that maps our AAA data to the data that must exist in `tailf-aaa.yang` and `ietf-netconf-acm.yang`. This way we can choose to export and and expose an entirely different AAA model. - -Yet another very easy way out, is to define a set of static AAA rules whereby a set of fixed users and fixed groups have fixed access to our configuration data. Possibly the only field we wish to manipulate is the password field. diff --git a/administration/management/high-availability.md b/administration/management/high-availability.md deleted file mode 100644 index b1272fce..00000000 --- a/administration/management/high-availability.md +++ /dev/null @@ -1,1236 +0,0 @@ ---- -description: Implement redundancy in your deployment using High Availability (HA) setup. ---- - -# High Availability - -As a single NSO node can fail or lose network connectivity, you can configure multiple nodes in a highly available (HA) setup, which replicates the CDB configuration and operational data across participating nodes. It allows the system to continue functioning even when some nodes are inoperable. - -The replication architecture is that of one active primary and a number of secondaries. This means all configuration write operations must occur on the primary, which distributes the updates to the secondaries. - -Operational data in the CDB may be replicated or not based on the `tailf:persistent` statement in the data model. If replicated, operational data writes can only be performed on the primary, whereas non-replicated operational data can also be written on the secondaries. - -Replication is supported in several different architectural setups. For example, two-node active/standby designs as well as multi-node clusters with runtime software upgrade. - -

Primary - Secondary Configuration

- -

One Primary - Several Secondaries

- -This feature is independent of but compatible with the [Layered Service Architecture (LSA)](../advanced-topics/layered-service-architecture.md), which also configures multiple NSO nodes to provide additional scalability. When the following text simply refers to a cluster, it identifies the set of NSO nodes participating in the same HA group, not an LSA cluster, which is a separate concept. - -NSO supports the following options for implementing an HA setup to cater to the widest possible range of use cases (only one can be used at a time): - -* [**HA Raft**](high-availability.md#ug.ha.raft): Using a modern, consensus-based algorithm, it offers a robust, hands-off solution that works best in the majority of cases. -* [**Rule-based HA**](high-availability.md#ug.ha.builtin): A less sophisticated solution that allows you to influence the primary selection but may require occasional manual operator action. -* [**External HA**](high-availability.md#ferret): NSO only provides data replication; all other functions, such as primary selection and group membership management, are performed by an external application, using the HA framework (HAFW). - -In addition to data replication, having a fixed address to connect to the current primary in an HA group greatly simplifies access for operators, users, and other systems alike. Use [Tail-f HCC Package](high-availability.md#ug.ha.hcc) or an [external load balancer](high-availability.md#ug.ha.lb) to manage it. - -## NSO HA Raft - -[Raft](https://raft.github.io/) is a consensus algorithm that reliably distributes a set of changes to a group of nodes and robustly handles network and node failure. It can operate in the face of multiple, subsequent failures, while also allowing a previously failed or disconnected node to automatically rejoin the cluster without risk of data conflicts. - -Compared to traditional fail-over HA solutions, Raft relies on the consensus of the participating nodes, which addresses the so-called “split-brain” problem, where multiple nodes assume a primary role. This problem is especially characteristic of two-node systems, where it is impossible for a single node on its own to distinguish between losing network connectivity itself versus the other node malfunctioning. For this reason, Raft requires at least three nodes in the cluster. - -Raft achieves robustness by requiring at least three nodes in the HA cluster. Three is the recommended cluster size, allowing the cluster to operate in the face of a single node failure. In case you need to tolerate two nodes failing simultaneously, you can add two additional nodes, for a 5-node cluster. However, permanently having more than five nodes in a single cluster is currently not recommended since Raft requires the majority of the currently configured nodes in the cluster to reach consensus. Without the consensus, the cluster cannot function. - -You can start a sample HA Raft cluster using the [examples.ncs/high-availability/raft-cluster](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/raft-cluster) example to test it out. The scripts in the example show various aspects of cluster setup and operation, which are further described in the rest of this section. - -Optionally, examples using separate containers for each HA Raft cluster member with NSO system installations are available and referenced in the [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc) example in the NSO example set. - -### Overview of Raft Operation - -The Raft algorithm works with the concept of (election) terms. In each term, nodes in the cluster vote for a leader. The leader is elected when it receives the majority of the votes. Since each node only votes for a single leader in a given term, there can only be one leader in the cluster for this term. - -Once elected, the leader becomes responsible for distributing the changes and ensuring consensus in the cluster for that term. Consensus means that the majority of the participating nodes must confirm a change before it is accepted. This is required for the system to ensure no changes ever get overwritten and provide reliability guarantees. On the other hand, it also means more than half of the nodes must be available for normal operation. - -Changes can only be performed on the leader, that will accept the change after the majority of the cluster nodes confirm it. This is the reason a typical Raft cluster has an odd number of nodes; exactly half of the nodes agreeing on a change is not sufficient. It also makes a two-node cluster (or any even number of nodes in a cluster) impractical; the system as a whole is no more available than it is with one fewer node. - -If the connection to the leader is broken, such as during a network partition, the nodes start a new term and a new election. Another node can become a leader if it gets the majority of the votes of all nodes initially in the cluster. While gathering votes, the node has the status of a candidate. In case multiple nodes assume candidate status, a split-vote scenario may occur, which is resolved by starting a fresh election until a candidate secures the majority vote. - -If it happens that there aren't enough reachable nodes to obtain a majority, a candidate can stay in the candidate state for an indefinite time. Otherwise, when a node votes for a candidate, it becomes a follower and stays a follower in this term, regardless if the candidate is elected or not. - -Additionally, the NSO node can also be in the stalled state, if HA Raft is enabled but the node has not joined a cluster. - -### Node Names and Certificates - -Each node in an HA Raft cluster needs a unique name. Names are usually in the `ADDRESS` format, where `ADDRESS` identifies a network host where the NSO process is running, such as a fully qualified domain name (FQDN) or an IPv4 address. - -Other nodes in the cluster must be able to resolve and reach the `ADDRESS`, which creates a dependency on the DNS if you use domain names instead of IP addresses. - -Limitations of the underlying platform place a constraint on the format of `ADDRESS`, which can't be a simple short name (without a dot), even if the system is able to resolve such a name using `hosts` file or a similar mechanism. - -You specify the node address in the `ncs.conf` file as the value for `node-address`, under the `listen` container. You can also use the full node name (with the “@” character), however, that is usually unnecessary as the system prepends `ncsd@` as-needed. - -Another aspect in which `ADDRESS` plays a role is authentication. The HA system uses mutual TLS to secure communication between cluster nodes. This requires you to configure a trusted Certificate Authority (CA) and a key/certificate pair for each node. When nodes connect, they check that the certificate of the peer validates against the CA and matches the `ADDRESS` of the peer. - -{% hint style="info" %} -Consider that TLS not only verifies that the certificate/key pair comes from a trusted source (certificate is signed by a trusted CA), it also checks that the certificate matches the host you are connecting to. Host A may have a valid certificate and key, signed by a trusted CA, however, if the certificate is for another host, say host B, the authentication will fail. -{% endhint %} - -In most cases, this means the `ADDRESS` must appear in the node certificate's Subject Alternative Name (SAN) extension, as `dNSName` (see [RFC2459](https://datatracker.ietf.org/doc/html/rfc2459)). - -Create and use a self-signed CA to secure the NSO HA Raft cluster. A self-signed CA is the only secure option. The CA should only be used to sign the certificates of the member nodes in one NSO HA Raft cluster. It is critical for security that the CA is not used to sign any other certificates. Any certificate signed by the CA can be used to gain complete control of the NSO HA Raft cluster. - -See the [examples.ncs/high-availability/raft-cluster](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/raft-cluster) example for one way to set up a self-signed CA and provision individual node certificates. The example uses a shell script `gen_tls_certs.sh` that invokes the `openssl` command. Consult the section [Recipe for a Self-signed CA](high-availability.md#recipe-for-a-self-signed-ca) for using it independently of the example. - -Examples using separate containers for each HA Raft cluster member with NSO system installations that use a variant of the `gen_tls_certs.sh` script are available and referenced in the [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc) example in the NSO example set. - -{% hint style="info" %} -When using an IP address instead of a DNS name for node's `ADDRESS`, you must add the IP address to the certificate's dNSName SAN field (adding it to iPAddress field only is insufficient). This is a known limitation in the current version. -{% endhint %} - -The following is a HA Raft configuration snippet for `ncs.conf` that includes certificate settings and a sample `ADDRESS`: - -```xml - - - - 198.51.100.10 - - - ${NCS_CONFIG_DIR}/dist/ssl/cert/myca.crt - ${NCS_CONFIG_DIR}/dist/ssl/cert/node-100-10.crt - ${NCS_CONFIG_DIR}/dist/ssl/cert/node-100-10.key - - -``` - -### Recipe for a Self-signed CA - -HA Raft uses a standard TLS protocol with public key cryptography for securing cross-node communication, where each node requires a separate public/private key pair and a corresponding certificate. Key and certificate management is a broad topic and is critical to the overall security of the system. - -The following text provides a recipe for generating certificates using a self-signed CA. It uses strong cryptography and algorithms that are deemed suitable for production use. However, it makes a few assumptions that may not be appropriate for all environments. Always consider how they affect your own deployment and consult a security professional if in doubt. - -The recipe makes the following assumptions: - -* You use a secured workstation or server to run these commands and handle the generated keys with care. In particular, you must copy the generated keys to NSO nodes in a secure fashion, such as using `scp`. -* The CA is used solely for a single NSO HA Raft cluster, with certificates valid for 10 years, and provides no CRL. If a single key or host is compromised, a new CA and all key/certificate pairs must be recreated and reprovisioned in the cluster. -* Keys and signatures based on ecdsa-with-sha384/P-384 are sufficiently secure for the vast majority of environments. However, if your organization has specific requirements, be sure to follow those. - -To use this recipe: - -* First prepare a working environment on a secure host by creating a new directory and copying the `gen_tls_certs.sh` script from [examples.ncs/high-availability/raft-cluster](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/raft-cluster) into it. Additionally, ensure that the `openssl` command, version 1.1 or later, is available and the system time is set correctly. Supposing that you have a cluster named `lower-west`, you might run: - -```bash -$ mkdir raft-ca-lower-west -$ cd raft-ca-lower-west -$ cp $NCS_DIR/examples.ncs/high-availability/raft-cluster/gen_tls_certs.sh . -$ openssl version -$ date -``` - -{% hint style="info" %} -Including cluster name in the directory name helps distinguish certificates of one HA cluster from another, such as when using an LSA deployment in an HA configuration. -{% endhint %} - -The recipe relies on the `gen_tls_certs.sh` script to generate individual certificates. For clusters using FQDN node addresses, invoke the script with full hostnames of all the participating nodes. For example: - -```bash -$ ./gen_tls_certs.sh node1.example.org node2.example.org node3.example.org -``` - -{% hint style="info" %} -Using only hostnames, e.g. `node1`, will not work. -{% endhint %} - -If your HA cluster is using IP addresses instead, add the `-a` option to the command and list the IPs: - -```bash -$ ./gen_tls_certs.sh -a 192.0.2.1 192.0.2.2 192.0.2.3 -``` - -The script outputs the location of the relevant files and you should securely transfer each set of files to the corresponding NSO node. For each node, transfer only the three files: `ca.crt`, _`host`_`.crt`, and _`host`_`.key`. - -* Once the certificates are deployed, you can check their validity with the `openssl verify` command: - -```bash -$ openssl verify -CAfile ssl/certs/ca.crt ssl/certs/node1.example.org.crt -``` - -This command takes into account the current time and can be used during troubleshooting. It can also display information contained in the certificate if you use the `openssl x509 -text -in ssl/certs/`_`node1.example.org`_`.crt -noout` variant. The latter form allows you to inspect the incorporated hostname/IP address and certificate validity dates. - -### Actions - -NSO HA Raft can be controlled through several actions. All actions are found under `/ha-raft/`. In the best-case scenario, you will only need the `create-cluster` action to initialize the cluster and the `read-only` and `create-cluster` actions when upgrading the NSO version. - -The available actions are listed below: - -
ActionDescription
create-clusterInitialize an HA Raft cluster. This action should only be invoked once to form a new cluster when no HA Raft log exists.
The members of the HA Raft cluster consist of the NCS node where the /ha-raft/create-clusteraction is invoked, which will become the leader of the cluster; and the members specified by the member parameter.
adjust-membershipAdd or remove an HA node from the HA Raft cluster.
disconnectDisconnect an HA node from all remaining nodes. In the event of revoking a TLS certificate, invoke this action to disconnect the already established connections to the node with the revoked certificate. A disconnected node with a valid TLS certificate may re-establish the connection.
resetReset the (disabled) local node to make the leader perform a full sync to this local node if an HA Raft cluster exists. If reset is performed on the leader node, the node will step down from leadership and it will be synced by the next leader node.
An HA Raft member will change role to disabled if ncs.conf has incompatible changes to the ncs.conf on the leader; a member will also change role to disabled if there are non-recoverable failures upon opening a snapshot.
See the /ha-raft/status/disable-reason leaf for the reason.
Set force to true to override reset when /ha-raft/status/role is not set to disabled.
handoverHandover leadership to another member of the HA Raft cluster or step down from leadership and start a new election.
read-onlyToggle read-only mode. If the mode is true no configuration changes can occur.
- -### Network and `ncs.conf` Prerequisites - -In addition to the network connectivity required for the normal operation of a standalone NSO node, nodes in the HA Raft cluster must be able to initiate TCP connections from a random ephemeral client port to the following ports on other nodes: - -* Port 4369 -* Ports in the range 4370-4399 (configurable) - -You can change the ports in the second listed range from the default of 4370-4399. Use the `min-port` and `max-port` settings of the `ha-raft/listen` container. - -The Raft implementation does not impose any other hard limits on the network but you should keep in mind that consensus requires communication with other nodes in the cluster. A high round-trip latency between cluster nodes is likely to negatively impact the transaction throughput of the system. - -The HA Raft cluster also requires compatible `ncs.conf` files among the member nodes. In particular, `/ncs-config/cdb/operational/enabled` and `/ncs-config/rollback/enabled` values affect replication behavior and must match. Likewise, each member must have the same set of encryption keys and the keys cannot be changed while the cluster is in operation. - -To update the `ncs.conf` configuration, you must manually update the copy on each member node, making sure the new versions contain compatible values. Then perform the reload on the leader and the follower members will automatically reload their copies of the configuration file as well. - -If a node is a cluster member but has been configured with a new, incompatible `ncs.conf` file, it gets automatically disabled. See the `/ha-raft/status/disabled-reason` for reason. You can re-enable the node with the `ha-raft reset` command, once you have reconciled the incompatibilities. - -### Connected Nodes and Node Discovery - -Raft has a notion of cluster configuration, in particular, how many and which members the cluster has. You define member nodes when you first initialize the cluster with the `create-cluster` command or use the `adjust-membership` command. The member nodes allow the cluster to know how many nodes are needed for consensus and similar. - -However, not all cluster members may be reachable or alive all the time. Raft implementation in NSO uses TCP connections between nodes to transport data. The TCP connections are authenticated and encrypted using TLS by default (see [Security Considerations](high-availability.md#ch_ha.raft_security)). A working connection between nodes is essential for the cluster to function but a number of factors, such as firewall rules or expired/invalid certificates, can prevent the connection from establishing. - -Therefore, NSO distinguishes between configured member nodes and nodes to which it has established a working transport connection. The latter are called connected nodes. In a normal, fully working, and properly configured cluster, the connected nodes will be the same as member nodes (except for the current node). - -To help troubleshoot connectivity issues without affecting cluster operation, connected nodes will show even nodes that are not actively participating in the cluster but have established a transport connection to nodes in the cluster. The optional discovery mechanism, described next, relies on this functionality. - -NSO includes a mechanism that simplifies the initial cluster setup by enumerating known nodes. This mechanism uses a set of seed nodes to discover all connectable nodes, which can then be used with the `create-cluster` command to form a Raft cluster. - -When you specify one or more nodes with the `/ha-raft/seed-nodes/seed-node` setting in the `ncs.conf` file, the current node tries to establish a connection to these seed nodes, in order to discover the list of all nodes potentially participating in the cluster. For the discovery to work properly, all other nodes must also use seed nodes and the set of seed nodes must overlap. The recommended practice is to use the same set of seed nodes on every participating node. - -Along with providing an autocompletion list for the `create-cluster` command, this feature streamlines the discovery of node names when using NSO in containerized or other dynamic environments, where node addresses are not known in advance. - -### Initial Cluster Setup - -Creating a new HA cluster consists of two parts: configuring the individual nodes and running the `create-cluster` action. - -First, you must update the `ncs.conf` configuration file for each node. All HA Raft configuration comes under the `/ncs-config/ha-raft` element. - -As part of the configuration, you must: - -* Enable HA Raft functionality through the `enabled` leaf. -* Set `node-address` and the corresponding TLS parameters (see [Node Names and Certificates](high-availability.md#ch_ha.raft_names)). -* Identify the cluster this node belongs to with `cluster-name`. -* Reload or restart the NSO process (if already running). -* Repeat the preceding steps for every participating node. -* Enable read-only mode on designated leader to avoid potential sync issues in cluster formation. -* Invoke the `create-cluster` action. - -The cluster name is simply a character string that uniquely identifies this HA cluster. The nodes in the cluster must use the same cluster name or they will refuse to establish a connection. This setting helps prevent mistakenly adding a node to the wrong cluster when multiple clusters are in operation, such as in an LSA setup. - -{% code title="Sample HA Raft config for a cluster node" %} -```xml - - true - sherwood - - ash.example.org - - - ${NCS_CONFIG_DIR}/dist/ssl/cert/myca.crt - ${NCS_CONFIG_DIR}/dist/ssl/cert/ash.crt - ${NCS_CONFIG_DIR}/dist/ssl/cert/ash.key - - - birch.example.org - - -``` -{% endcode %} - -With all the nodes configured and running, connect to the node that you would like to become the initial leader and invoke the `ha-raft create-cluster` action. The action takes a list of nodes identified by their names. If you have configured `seed-nodes`, you will get auto-completion support, otherwise, you have to type in the names of the nodes yourself. - -This action makes the current node a cluster leader and joins the other specified nodes to the newly created cluster. For example: - -```bash -admin@ncs# request ha-raft read-only-mode true -admin@ncs# ha-raft create-cluster member [ birch.example.org cedar.example.org ] -admin@ncs# show ha-raft -ha-raft status role leader -ha-raft status leader ash.example.org -ha-raft status member [ ash.example.org birch.example.org cedar.example.org ] -ha-raft status connected-node [ birch.example.org cedar.example.org ] -ha-raft status local-node ash.example.org -... -admin@ncs# request ha-raft read-only-mode false -``` - -You can use the `show ha-raft` command on any node to inspect the status of the HA Raft cluster. The output includes the current cluster leader and members according to this node, as well as information about the local node, such as node name (`local-node`) and role. The `status/connected-node` list contains the names of the nodes with which this node has active network connections. - -
- -show ha-raft Field Definitions - -The command `show ha-raft` is used in NSO to display the current state of the HA Raft cluster. The output typically includes the following information: - -* The role of the local node (for example, whether it is the `leader`, `follower`, `candidate`, or `stalled`). -* The leader of the cluster, if one has been elected. -* The list of member nodes that belong to the HA Raft cluster. -* The connected nodes, which are the nodes with which the local node currently has active RAFT communication. -* The local node information, detailing the node’s name and status. - -This command is useful for both verifying that the HA Raft cluster is set up correctly and for troubleshooting issues by checking the connectivity and role assignments of the nodes. Some noteworthy terms of output are defined in the table below. - -
TermDefinition
roleThe current node’s Raft role (leader, follower, or candidate). Occasionally, in NSO, a node might appear as stalled if it has lost contact with the leader or quorum.
leaderThe current known leader of the cluster.
memberA node that is part of the RAFT consensus group (i.e., a voting participant, not an observer). Leaders, followers, and candidates are members; observers are not.
connected-nodeThe nodes this instance is connected to.
local-nodeThe name of the current node.
lagThe number of indices the replicated log is behind the leader node. A value of 0 means no lag — the node's RAFT log is fully up-to-date with the leader. The larger the value, the more out-of-sync the node is, which may indicate a replication or connectivity issue.
indexThe last replicated HA Raft log index, i.e., this is the last log entry replicated to a node.
state

The synchronization status of the node’s RAFT log. Common values include:

  • in-sync: The node is up-to-date with the leader.
  • behind: The node is lagging behind in log replication.
  • unreachable: The node is not communicating with one or more RAFT peers, i.e., the node cannot reach the leader or other RAFT peers, preventing synchronization.
  • requires-snapshot: The node has fallen too far behind to catch up using logs and needs a full snapshot from the leader.
current-indexThe latest log index on this node.
applied-indexThe last index applied to CDB.
serial-numberThe certificate serial number. Used to uniquely identify the node.
- -
- -In case you get an error, such as the `Error: NSO can't reach member node 'ncsd@ADDRESS'.`, please verify all of the following: - -* The node at the `ADDRESS` is reachable. You can use the `ping` `ADDRESS` command, for example. -* The problematic node has the correct `ncs.conf` configuration, especially `cluster-name` and `node-address`. The latter should match the `ADDRESS` and should contain at least one dot. -* Nodes use compatible configuration. For example, make sure the `ncs.crypto_keys` file (if used) or the `encrypted-strings` configuration in `ncs.conf` is identical across nodes. -* HA Raft is enabled, using the `show ha-raft` command on the unreachable node. -* The firewall configuration on the OS and on the network level permits traffic on the required ports (see [Network and `ncs.conf` Prerequisites](high-availability.md#ch_ha.raft_ports)). -* The node uses a certificate that the CA can validate. For example, copy the certificates to the same location and run `openssl verify -CAfile CA_CERT NODE_CERT` to verify this. -* Verify the `epmd -names` command on each node shows the ncsd process. If not, stop NSO, run `epmd -kill`, and then start NSO again. - -In addition to the above, you may also examine the `logs/raft.log` file for detailed information on the error message and overall operation of the Raft algorithm. The amount of information in the file is controlled by the `/ncs-config/logs/raft-log` configuration in the `ncs.conf`. - -### Cluster Management - -After the initial cluster setup, you can add new nodes or remove existing nodes from the cluster with the help of the `ha-raft adjust-membership` action. For example: - -```bash -admin@ncs# show ha-raft status member -ha-raft status member [ ash.example.org birch.example.org cedar.example.org ] -admin@ncs# ha-raft adjust-membership remove-node birch.example.org -admin@ncs# show ha-raft status member -ha-raft status member [ ash.example.org cedar.example.org ] -admin@ncs# ha-raft adjust-membership add-node dollartree.example.org -admin@ncs# show ha-raft status member -ha-raft status member [ ash.example.org cedar.example.org dollartree.example.org ] -``` - -When removing nodes using the `ha-raft adjust-membership remove-node` command, the removed node is not made aware that it is removed from the cluster and continues signaling the other nodes. This is a limitation in the algorithm, as it must also handle situations, where the removed node is down or unreachable. To prevent further communication with the cluster, it is important you ensure the removed node is shut down. You should shut down the to-be-removed node prior to removal from the cluster, or immediately after it. The former is recommended but the latter is required if there are only two nodes left in the cluster and shutting down prior to removal would prevent the cluster from reaching consensus. - -Additionally, you can force an existing follower node to perform a full re-sync from the leader by invoking the `ha-raft reset` action with the `force` option. Using this action on the leader will make the node give up the leader role and perform a sync with the newly elected leader. - -As leader selection during the Raft election is not deterministic, NSO provides the `ha-raft handover` action, which allows you to either trigger a new election if called with no arguments or transfer leadership to a specific node. The latter is especially useful when, for example, one of the nodes resides in a different location and more traffic between locations may incur extra costs or additional latency, so you prefer this node is not the leader under normal conditions. - -#### Passive Follower - -In certain situations, it may be advantageous to have a follower node that cannot be promoted to leader role. Consider a scenario with three Raft-enabled nodes distributed across two different data centers. - -In this case, a node located without a peer in the same data center might experience increased latency due to the requirement for acknowledgments from at least one node in the other data center. - -To address this, HA Raft provides the `/ncs-config/ha-raft/passive` setting. When this setting is enabled (set to `true`), it prevents the node from assuming the candidate or leader role. A passive follower still participates by voting in leader elections. - -Note that the `passive` parameter is local to the node, meaning other nodes in the cluster are unaware that a particular follower is passive. Consequently, it is possible to initiate a handover action targeting the passive node, but the handover will ultimately fail at a later stage, allowing the current leader to retain its position. - -### Migrating From Existing Rule-based HA - -If you have an existing HA cluster using the rule-based built-in HA, you can migrate it to use HA Raft instead. This procedure is performed in four distinct high-level steps: - -* Ensuring the existing cluster meets migration prerequisites. -* Preparing the required HA Raft configuration files. -* Switching to HA Raft. -* Adding additional nodes to the cluster. - -The procedure does not perform an NSO version upgrade, so the cluster remains on the same version. It also does not perform any schema upgrades, it only changes the type of the HA cluster. - -The migration procedure is in place, that is, the existing nodes are disconnected from the old cluster and connected to the new one. This results in a temporary disruption of the service, so it should be performed during a service window. - -First, you should ensure the cluster meets migration prerequisites. The cluster must use: - -* NSO 6.1.2 or later -* tailf-hcc 6.0 or later (if used) - -In case these prerequisites are not met, follow the standard upgrade procedures to upgrade the existing cluster to supported versions first. - -Additionally, ensure that all used packages are compatible with HA Raft, as NSO uses some new or updated notifications about HA state changes. Also, verify the network supports the new cluster communications (see [Network and `ncs.conf` Prerequisites](high-availability.md#ch_ha.raft_ports)). - -Secondly, prepare all the `ncs.conf` and related files for each node, such as certificates and keys. Create a copy of all the `ncs.conf` files and disable or remove the existing `>ha<` section in the copies. Then add the required configuration items to the copies, as described in [Initial Cluster Setup](high-availability.md#ch_ha.raft_setup) and [Node Names and Certificates](high-availability.md#ch_ha.raft_names). Do not update the `ncs.conf` files used by the nodes yet. - -It is recommended but not necessary that you set the seed nodes in `ncs.conf` to the designated primary and fail-over primary. Do this for all `ncs.conf` files for all nodes. - -#### Procedure 1. Migration to HA Raft - -1. With the new configurations at hand and verified, start the switch to HA Raft. The cluster nodes should be in their nominal, designated roles. If not, perform a failover first. -2. On the designated (actual) primary, called `node1`, enable read-only mode. - - ```bash - admin@node1# high-availability read-only mode true - ``` -3. Then take a backup of all nodes. -4. Once the backup successfully completes, stop the designated fail-over primary (actual secondary) NSO process, update its `ncs.conf` and the related (certificate) files for HA Raft, and then start it again. Connect to this node's CLI, here called node2, and verify HA Raft is enabled with the `show` `ha-raft` command. - - ```bash - admin@node2# show ha-raft - ha-raft status role stalled - ha-raft status local-node node2.example.org - > ... output omitted ... < - ``` -5. Now repeat the same for the designated primary (`node1`). If you have set the seed nodes, you should see the fail-over primary show under `connected-node`. - - ```bash - admin@node1# show ha-raft - ha-raft status role stalled - ha-raft status connected-node [ node2.example.org ] - ha-raft status local-node node1.example.org - > ... output omitted ... < - ``` -6. On the old designated primary (node1) invoke the `ha-raft create-cluster` action and create a two-node Raft cluster with the old fail-over primary (`node2`, actual secondary). The action takes a list of nodes identified by their names. If you have configured `seed-nodes`, you will get auto-completion support, otherwise you have to type in the name of the node yourself. - - ```bash - admin@node1# ha-raft create-cluster member [ node2.example.org ] - admin@node1# show ha-raft - ha-raft status role leader - ha-raft status leader node1.example.org - ha-raft status member [ node1.example.org node2.example.org ] - ha-raft status connected-node [ node2.example.org ] - ha-raft status local-node node1.example.org - > ... output omitted ... < - ``` - - In case of errors running the action, refer to [Initial Cluster Setup](high-availability.md#ch_ha.raft_setup) for possible causes and troubleshooting steps. -7. Raft requires at least three nodes to operate effectively (as described in [NSO HA Raft](high-availability.md#ug.ha.raft)) and currently, there are only two in the cluster. If the initial cluster had only two nodes, you must provision an additional node and set it up for HA Raft. If the cluster initially had three nodes, there is the remaining secondary node, `node3`, which you must stop, update its configuration as you did with the other two nodes, and start it up again. -8. Finally, on the old designated primary and current HA Raft leader, use the `ha-raft adjust-membership add-node` action to add this third node to the cluster. - - ```bash - admin@node1# ha-raft adjust-membership add-node node3.example.org - admin@node1# show ha-raft status member - ha-raft status member [ node1.example.org node2.example.org node3.example.org ] - ``` - -### Security Considerations - -Communication between the NSO nodes in an HA Raft cluster takes place over Distributed Erlang, an RPC protocol transported over TLS (unless explicitly disabled by setting `/ncs-config/ha-raft/ssl/enabled` to 'false'). - -TLS (Transport Layer Security) provides Authentication and Privacy by only allowing NSO nodes to connect using certificates and keys issued from the same Certificate Authority (CA). Distributed Erlang is transported over TLS 1.2. Access to a host can be revoked by the CA through the means of a CRL (Certificate Revocation List). To enforce certificate revocation within an HA Raft cluster, invoke the action /ha-raft/disconnect to terminate the pre-existing connection. A connection to the node can re-establish once the node's certificate is valid. - -Please ensure the CA key is kept in a safe place since it can be used to generate new certificates and key pairs for peers. - -Distributed Erlang supports for multiple NSO nodes to run on the same host and the node addresses are resolved by the `epmd` ([Erlang Port Mapper Daemon](https://www.erlang.org/resources/man/epmd.html)) service. Once resolved, the NSO nodes communicate directly. - -The ports `epmd` and the NSO nodes listen to can be found in [Network and `ncs.conf` Prerequisites](high-availability.md#ch_ha.raft_ports). `epmd` binds the wildcard IPv4 address `0.0.0.0` and the IPv6 address `::`. - -In case `epmd` is exposed to a DoS attack, the HA Raft members may be unable to resolve addresses and communication could be disrupted. Please ensure traffic on these ports are only accepted between the HA Raft members by using firewall rules or other means. - -Two NSO nodes can only establish a connection if a shared secret "cookie" matches. The cookie is optionally configured from `/ncs-config/ha-raft/cluster-name`. Please note the cookie is not a security feature but a way to isolate HA Raft clusters and to avoid accidental misuse. - -### Packages Upgrades in Raft Cluster - -NSO contains a mechanism for distributing packages to nodes in a Raft cluster, greatly simplifying package management in a highly-available setup. - -You perform all package management operations on the current leader node. To identify the leader node, you can use the `show ha-raft status leader` command on a running cluster. - -Invoking the `packages reload` command makes the leader node update its currently loaded packages, identical to a non-HA, single-node setup. At the same time, the leader also distributes these packages to the followers to load. However, the load paths on the follower nodes, such as `/var/opt/ncs/packages/`, are not updated. This means, that if a leader election took place, a different leader was elected, and you performed another `packages reload`, the system would try to load the versions of the packages on this other leader, which may be out of date or not even present. - -The recommended approach is, therefore, to use the `packages ha sync and-reload` command instead, unless a load path is shared between NSO nodes, such as the same network drive. This command distributes and updates packages in the load paths on the follower nodes, as well as loading them. - -For the full procedure, first, ensure all cluster nodes are up and operational, then follow these steps on the leader node: - -* Perform a full backup of the NSO instance, such as running `ncs-backup`. -* Add, replace, or remove packages on the filesystem. The exact location depends on the type of NSO deployment, for example `/var/opt/ncs/packages/`. -* Invoke the `packages ha sync and-reload` or `packages ha sync and-add` command to start the upgrade process. - -Note that while the upgrade is in progress, writes to the CDB are not allowed and will be rejected. - -For a `packages ha sync and-reload` example see the `raft-upgrade-l2` NSO system installation-based example referenced by the [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc) example in the NSO example set. - -For more details, troubleshooting, and general upgrade recommendations, see [NSO Packages](package-mgmt.md) and [Upgrade](../installation-and-deployment/upgrade-nso.md). - -### Version Upgrade of Cluster Nodes - -Currently, the only supported and safe way of upgrading the Raft HA cluster NSO version requires that the cluster be taken offline since the nodes must, at all times, run the same software version. - -Do not attempt an upgrade unless all cluster member nodes are up and actively participating in the cluster. Verify the current cluster state with the `show ha-raft status` command. All member nodes must also be present in the connected-node list. - -The procedure differentiates between the current leader node versus followers. To identify the leader, you can use the `show ha-raft status leader` command on a running cluster. - -**Procedure 2. Cluster Version Upgrade** - -1. On the leader, first enable read-only mode using the `ha-raft read-only mode true` command and then verify that all cluster nodes are in sync with the `show ha-raft status log replications state` command. -2. Before embarking on the upgrade procedure, it's imperative to backup each node. This ensures that you have a safety net in case of any unforeseen issues. For example, you can use the `$NCS_DIR/bin/ncs-backup` command. -3. Delete the `$NCS_RUN_DIR/cdb/compact.lock` file and compact the CDB write log on all nodes using, for example, the `$NCS_DIR/bin/ncs --cdb-compact $NCS_RUN_DIR/cdb` command. -4. On all nodes, delete the `$NCS_RUN_DIR/state/raft/` directory with a command such as `rm -rf $NCS_RUN_DIR/state/raft/`. -5. Stop NSO on all the follower nodes, for example, invoking the `$NCS_DIR/bin/ncs --stop` or `systemctl stop ncs` command on each node. -6. Stop NSO on the leader node only after you have stopped all the follower nodes in the previous step. Alternatively NSO can be stopped on the nodes before deleting the HA Raft state and compacting the CDB write log without needing to delete the `compact.lock` file. -7. Upgrade the NSO packages on the leader to support the new NSO version. -8. Install the new NSO version on all nodes. -9. Start NSO on all nodes. -10. Re-initialize the HA cluster using the `ha-raft create-cluster` action on the node to become the leader. -11. Finally, verify the cluster's state through the `show ha-raft status` command. Ensure that all data has been correctly synchronized across all cluster nodes and that the leader is no longer read-only. The latter happens automatically after re-initializing the HA cluster. - -For a standard System Install, the single-node procedure is described in [Single Instance Upgrade](../installation-and-deployment/upgrade-nso.md#ug.admin_guide.manual_upgrade), but in general depends on the NSO deployment type. For example, it will be different for containerized environments. For specifics, please refer to the documentation for the deployment type. - -For an example see the `raft-upgrade-l2` NSO system installation-based example referenced by the [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc) example in the NSO example set. - -If the upgrade fails before or during the upgrade of the original leader, start up the original followers to restore service and then restore the original leader, using backup as necessary. - -However, if the upgrade fails after the original leader was successfully upgraded, you should still be able to complete the cluster upgrade. If you are unable to upgrade a follower node, you may provision a (fresh) replacement and the data and packages in use will be copied from the leader. - -## NSO Rule-based HA - -NSO can manage the HA groups based on a set of predefined rules. This functionality was added in NSO 5.4 and is sometimes referred to simply as the built-in HA. However, since NSO 6.1, HA Raft (which is also built-in) is available as well, and is likely a better choice in most situations. - -Rule-based HA allows administrators to: - -* Configure HA group members with IP addresses and default roles -* Configure failover behavior -* Configure start-up behavior -* Configure HA group members with IP addresses and default roles -* Assign roles, join HA group, enable/disable rule-based HA through actions -* View the state of the current HA setup - -NSO rule-based HA is defined in `tailf-ncs-high-availability.yang`, with data residing under the `/high-availability/` container. - -{% hint style="info" %} -In environments with high NETCONF traffic, particularly when using `ncs_device_notifs`, it's recommended to enable read-only mode on the designated primary node before performing HA activation or sync. This prevents `app_sync` from being blocked by notification processing. - -Use the following command prior to enabling HA or assigning roles: - -```bash -admin@ncs# high-availability read-only mode true -``` - -After successful sync and HA establishment, disable read-only mode: - -```bash -admin@ncs# high-availability read-only mode false -``` -{% endhint %} - -NSO rule-based HA does not manage any virtual IP addresses, or advertise any BGP routes or similar. This must be handled by an external package. Tail-f HCC 5.x and greater has this functionality compatible with NSO rule-based HA. You can read more about the HCC package in the [following chapter](high-availability.md#ug.ha.hcc). - -### Prerequisites - -To use NSO rule-based HA, HA must first be enabled in `ncs.conf` - See [Mode of Operation](high-availability.md#ha.moo). - -{% hint style="info" %} -If the package tailf-hcc with a version less than 5.0 is loaded, NSO rule-based HA will not function. These HCC versions may still be used but NSO built-in HA will not function in parallel. -{% endhint %} - -### HA Member Configuration - -All HA group members are defined under `/high-availability/ha-node`. Each configured node must have a unique IP address configured and a unique HA ID. Additionally, nominal roles and fail-over settings may be configured on a per-node basis. - -The HA Node ID is a unique identifier used to identify NSO instances in an HA group. The HA ID of the local node - relevant amongst others when an action is called - is determined by matching configured HA node IP addresses against IP addresses assigned to the host machine of the NSO instance. As the HA ID is crucial to NSO HA, NSO rule-based HA will not function if the local node cannot be identified. - -To join a HA group, a shared secret must be configured on the active primary and any prospective secondary. This is used for a CHAP-2-like authentication and is specified under `/high-availability/token/`. - -{% hint style="info" %} -In an NSO System Install setup, not only does the shared token need to match between the HA group nodes but the configuration for encrypted strings, default stored in `/etc/ncs/ncs.crypto_keys`, need also to match between the nodes in the HA group. -{% endhint %} - -The token configured on the secondary node is overwritten with the encrypted token of type `aes-256-cfb-128-encrypted-string` from the primary node when the secondary node connects to the primary. If there is a mismatch between the encrypted-string configuration on the nodes, NSO will not decrypt the HA token to match the token presented. As a result, the primary node denies the secondary node access the next time the HA connection needs to reestablish with a "Token mismatch, secondary is not allowed" error. - -See the `upgrade-l2` example, referenced from [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc), for an example setup and the [Deployment Example](../installation-and-deployment/deployment/deployment-example.md) for a description of the example. - -Also, note that the `ncs.crypto_keys` file is highly sensitive. The file contains the encryption keys for all CDB data that is encrypted on disk. Besides the HA token, this often includes passwords for various entities, such as login credentials to managed devices. - -### HA Roles - -NSO can assume HA roles `primary`, `secondary` and `none`. Roles can be assigned directly through actions, or at startup or failover. See [HA Framework Requirements](high-availability.md#ferret) for the definition of these roles. - -{% hint style="info" %} -NSO rule-based HA does not support relay-secondaries. -{% endhint %} - -NSO rule-based HA distinguishes between the concepts of nominal role and assigned role. Nominal-role is configuration data that applies when an NSO instance starts up and at failover. The assigned role is the role that the NSO instance has been ordered to assume either by an action or as a result of startup or failover. - -### Failover - -Failover may occur when a secondary node loses the connection to the primary node. A secondary may then take over the primary role. Failover behavior is configurable and controlled by the parameters: - -* `/high-availability/ha-node{id}/failover-primary` -* `/high-availability/settings/enable-failover` - -For automatic failover to function, `/high-availability/settings/enable-failover` must be se to `true`. It is then possible to enable at most one node with a nominal role secondary as failover-primary, by setting the parameter `/high-availability/ha-node{id}/failover-primary`. The failover works in both directions; if a nominal primary is currently connected to the failover-primary as a secondary and loses the connection, then it will attempt to take over as a primary. - -Before failover happens, a failover-primary-enabled secondary node may attempt to reconnect to the previous primary before assuming the primary role. This behavior is configured by the parameters denoting how many reconnect attempts will be made, and with which interval, respectively. - -* `/high-availability/settings/reconnect-attempts` -* `/high-availability/settings/reconnect-interval` - -HA Members that are assigned as secondaries, but are neither failover-primaries nor set with a nominal-role primary, may attempt to rejoin the HA group after losing connection to the primary. - -This is controlled by `/high-availability/settings/reconnect-secondaries`. If this is true, secondary nodes will query the nodes configured under `/high-availability/ha-node` for an NSO instance that currently has the primary role. Any configured nominal roles will not be considered. If no primary node is found, subsequent attempts to rejoin the HA setup will be issued with an interval defined by `/high-availability/settings/reconnect-interval`. - -In case a net-split provokes a failover it is possible to end up in a situation with two primaries, both nodes accepting writes. The primaries are then not synchronized and will end up in a split brain. Once one of the primaries joins the other as a secondary, the HA cluster is once again consistent but any out-of-sync changes will be overwritten. - -To prevent split-brain from occurring, NSO 5.7 or later comes with a rule-based algorithm. The algorithm is enabled by default, it can be disabled or changed from the parameters: - -* `/high-availability/settings/consensus/enabled [true]` -* `/high-availability/settings/consensus/algorithm [ncs:rule-based]` - -The rule-based algorithm can be used in either of the two HA constellations: - -* Two nodes: one nominal primary and one nominal secondary configured as failover-primary. -* Three nodes: one nominal primary, one nominal secondary configured as failover-primary, and one perpetual secondary. - -On failover: - -* Failover-primary: become primary but enable read-only mode. Once the secondary joins, disable read-only. -* Nominal primary: on loss of all secondaries, change role to none. If one secondary node is connected, stay primary. - -{% hint style="info" %} -In certain cases, the rule-based consensus algorithm results in nodes being disconnected and will not automatically rejoin the HA cluster, such as in the example above when the nominal primary becomes none on the loss of all secondaries. -{% endhint %} - -To restore the HA cluster one may need to manually invoke the `/high-availability/be-secondary-to` action. - -{% hint style="info" %} -In the case where the failover-primary takes over as primary, it will enable read-only mode, if no secondary connects it will remain read-only. This is done to guarantee consistency. -{% endhint %} - -{% hint style="info" %} -In a three-node cluster, when the nominal primary takes over as actual primary, it first enables read-only mode and stays in read-only mode until a secondary connects. This is done to guarantee consistency. -{% endhint %} - -The read-write mode can manually be enabled from the `/high-availability/read-only` action with the parameter mode passed with value false. - -When any node loses connection, this can also be observed in high-availability alarms as either a `ha-primary-down` or a `ha-secondary-down` alarm. - -```bash -alarms alarm-list alarm ncs ha-primary-down /high-availability/ha-node[id='paris'] - is-cleared false - last-status-change 2022-05-30T10:02:45.706947+00:00 - last-perceived-severity critical - last-alarm-text "Lost connection to primary due to: Primary closed connection" - status-change 2022-05-30T10:02:45.706947+00:00 - received-time 2022-05-30T10:02:45.706947+00:00 - perceived-severity critical - alarm-text "Lost connection to primary due to: Primary closed connection" -``` - -```bash -alarms alarm-list alarm ncs ha-secondary-down /high-availability/ha-node[id='london'] "" - is-cleared false - last-status-change 2022-05-30T10:04:33.231808+00:00 - last-perceived-severity critical - last-alarm-text "Lost connection to secondary" - status-change 2022-05-30T10:04:33.231808+00:00 - received-time 2022-05-30T10:04:33.231808+00:00 - perceived-severity critical - alarm-text "Lost connection to secondary" -``` - -### Startup - -Startup behavior is defined by a combination of the parameters `/high-availability/settings/start-up/assume-nominal-role` and `/high-availability/settings/start-up/join-ha` as well as the node's nominal role: - -
assume-nominal-rolejoin-hanominal-roleBehaviour
truefalseprimaryAssume primary role.
truefalsesecondaryAttempt to connect as secondary to the node (if any), which has nominal-role primary. If this fails, make no retry attempts and assume none role.
truefalsenoneAssume none role
falsetrueprimaryAttempt to join HA setup as secondary by querying for the current primary. Retries will be attempted. Retry attempt interval is defined by /high-availability/settings/reconnect-interval.
falsetruesecondaryAttempt to join HA setup as secondary by querying for the current primary. Retries will be attempted. Retry attempt interval is defined by /high-availability/settings/reconnect-interval. If all retry attempts fail, assume none role.
falsetruenoneAssume none role.
truetrueprimaryQuery HA setup once for a node with primary role. If found, attempt to connect as secondary to that node. If no current primary is found, assume primary role.
truetruesecondaryAttempt to join HA setup as secondary by querying for the current primary. Retries will be attempted. Retry attempt interval is defined by /high-availability/settings/reconnect-interval. If all retry attempts fail, assume none role.
truetruenoneAssume none role.
falsefalse-Assume none role.
- -### Actions - -NSO rule-based HA can be controlled through several actions. All actions are found under `/high-availability/`. The available actions are listed below: - -
ActionDescription
be-primaryOrder the local node to assume the HA role primary.
be-noneOrder the local node to assume the HA role none.
be-secondary-toOrder the local node to connect as secondary to the provided HA node. This is an asynchronous operation; the result can be found under /high-availability/status/be-secondary-result.
local-node-idIdentify which of the nodes in /high-availability/ha-node (if any) corresponds to the local NSO instance.
enableEnable NSO rule-based HA and optionally assume an HA role according to /high-availability/settings/start-up/ parameters.
disableDisable NSO rule-based HA and assume a HA role none.
- -### Status Check - -The current state of NSO rule-based HA can be monitored by observing `/high-availability/status/`. Information can be found about the current active HA mode and the current assigned role. For nodes with active mode primary, a list of connected nodes and their source IP addresses is shown. For nodes with assigned role secondary the latest result of the be-secondary operation is listed. All NSO rule-based HA status information is non-replicated operational data - the result here will differ between nodes connected in an HA setup. - -## Tail-f HCC Package - -The Tail-f HCC package extends the built-in HA functionality by providing virtual IP addresses (VIPs) that can be used to connect to the NSO HA group primary node. HCC ensures that the VIP addresses are always bound by the HA group primary and never bound by a secondary. Each time a node transitions between primary and secondary states HCC reacts by binding (primary) or unbinding (secondary) the VIP addresses. - -HCC manages IP addresses at the link layer (OSI layer 2) for Ethernet interfaces, and optionally, also at the network layer (OSI layer 3) using BGP router advertisements. The layer-2 and layer-3 functions are mostly independent and this document describes the details of each one separately. However, the layer-3 function builds on top of the layer-2 function. The layer-2 function is always necessary, otherwise, the Linux kernel on the primary node would not recognize the VIP address or accept traffic directed to it. - -{% hint style="info" %} -Tail-f HCC version 5.x is non-backward compatible with previous versions of Tail-f HCC and requires functionality provided by NSO version 5.4 and greater. For more details, see the [following chapter](high-availability.md#ug.ha.hcc.compared). -{% endhint %} - -### Dependencies - -Both the HCC layer-2 VIP and layer-3 BGP functionality depend on `iproute2` utilities and `awk`. An optional dependency is `arping` (either from `iputils` or Thomas Habets `arping` implementation) which allows HCC to announce the VIP to MAC mapping to all nodes in the network by sending gratuitous ARP requests. - -The HCC layer-3 BGP functionality depends on the [`GoBGP`](https://osrg.github.io/gobgp/) daemon version 2.x being installed on each NSO host that is configured to run HCC in BGP mode. - -GoBGP is open-source software originally developed by NTT Communications and released under the Apache License 2.0. GoBGP can be obtained directly from https://osrg.github.io/gobgp/ and is also packaged for mainstream Linux distributions. - -The HCC layer-3 DNS Update functionality depends on the command line utility `nsupdate`. - -Tools Dependencies are listed below: - -
ToolPackageRequiredDescription
ipiproute2yesAdds and deletes the virtual IP from the network interface.
awkmawk or gawkyesInstalled with most Linux distributions.
sedsedyesInstalled with most Linux distributions.
arpingiputils or arpingoptionalInstallation recommended. Will reduce the propagation of changes to the virtual IP for layer-2 configurations.
gobgpd and gobgpGoBGP 2.xoptionalRequired for layer-3 configurations. gobgpd is started by the HCC package and advertises the virtual IP using BGP. gobgp is used to get advertised routes.
nsupdatebind-tools or knot-dnsutilsoptionalRequired for layer-3 DNS update functionality and is used to submit Dynamic DNS Update requests to a name server.
- -Same as with built-in HA functionality, all NSO instances must be configured to run in HA mode. See the [following instructions](high-availability.md#ha.moo) on how to enable HA on NSO instances. - -### Running the HCC Package with NSO as a Non-Root User - -GoBGP uses TCP port 179 for its communications and binds to it at startup. As port 179 is considered a privileged port it is normally required to run gobgpd as root. - -When NSO is running as a non-root user the gobgpd command will be executed as the same user as NSO and will prevent gobgpd from binding to port 179. - -There a multiple ways of handling this and two are listed here. - -1. Set capability `CAP_NET_BIND_SERVICE` on the `gobgpd` file. May not be supported by all Linux distributions. - - ```bash - $ sudo setcap 'cap_net_bind_service=+ep' /usr/bin/gobgpd - ``` -2. Set the owner to `root` and the `setuid` bit of the `gobgpd` file. Works on all Linux distributions. - - ```bash - $ sudo chown root /usr/bin/gobgpd - $ sudo chmod u+s /usr/bin/gobgpd - ``` -3. The `vipctl` script, included in the HCC package, uses `sudo` to run the `ip` and `arping` commands when NSO is not running as root. If `sudo` is used, you must ensure it does not require password input. For example, if NSO runs as `admin` user, the `sudoers` file can be edited similarly to the following: - - ```bash - $ sudo echo "admin ALL = (root) NOPASSWD: /bin/ip" >> /etc/sudoers - $ sudo echo "admin ALL = (root) NOPASSWD: /path/to/arping" >> /etc/sudoers - ``` - -### Tail-f HCC Compared with HCC Version 4.x and Older - -#### **HA Group Management Decisions** - -Tail-f HCC 5.x or later does not participate in decisions on which NSO node is primary or secondary. These decisions are taken by NSO's built-in HA and then pushed as notifications to HCC. The NSO built-in HA functionality is available in NSO starting with version 5.4, where older NSO versions are not compatible with the HCC 5.x or later. - -#### **Embedded BGP Daemon** - -HCC 5.x or later operates a GoBGP daemon as a subprocess completely managed by NSO. The old HCC function pack interacted with an external Quagga BGP daemon using a NED interface. - -#### **Automatic Interface Assignment** - -HCC 5.x or later automatically associates VIP addresses with Linux network interfaces using the `ip` utility from the iproute2 package. VIP addresses are also treated as `/32` without defining a new subnet. The old HCC function pack used explicit configuration to associate VIPs with existing addresses on each NSO host and define IP subnets for VIP addresses. - -### Upgrading - -Since version 5.0, HCC relies on the NSO built-in HA for cluster management and only performs address or route management in reaction to cluster changes. Therefore, no special measures are necessary if using HCC when performing an NSO version upgrade or a package upgrade. Instead, you should follow the standard best practice HA upgrade procedure from [NSO HA Version Upgrade](../installation-and-deployment/upgrade-nso.md#ch_upgrade.ha). - -A reference to upgrade examples can be found in the README under [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc). - -### Layer-2 - -The purpose of the HCC layer-2 functionality is to ensure that the configured VIP addresses are bound in the Linux kernel of the NSO primary node only. This ensures that the primary node (and only the primary node) will accept traffic directed toward the VIP addresses. - -HCC also notifies the local layer-2 network when VIP addresses are bound by sending Gratuitous ARP (GARP) packets. Upon receiving the Gratuitous ARP, all the nodes in the network update their ARP tables with the new mapping so they can continue to send traffic to the non-failed, now primary node. - -#### **Operational Details** - -HCC binds the VIP addresses as additional (alias) addresses on existing Linux network interfaces (e.g. `eth0`). The network interface for each VIP is chosen automatically by performing a kernel routing lookup on the VIP address. That is, the VIP will automatically be associated with the same network interface that the Linux kernel chooses to send traffic to the VIP. - -This means that you can map each VIP onto a particular interface by defining a route for a subnet that includes the VIP. If no such specific route exists the VIP will automatically be mapped onto the interface of the default gateway. - -{% hint style="info" %} -To check which interface HCC will choose for a particular VIP address, simply run for example and look at the device `dev` in the output, for example `eth0`: - -```bash -admin@paris:~$ ip route get 192.168.123.22 -``` -{% endhint %} - -#### **Configuration** - -The layer-2 functionality is configured by providing a list of IPv4 and/or IPv6 VIP addresses and enabling HCC. The VIP configuration parameters are found under `/hcc:hcc`. - -Global Layer-2 Configuration: - -
ParametersTypeDescription
enabledbooleanIf set to 'true', the primary node in an HA group automatically binds the set of Virtual IPv[46] addresses.
vip-addresslist of inet:ip-addressThe list of virtual IPv[46] addresses to bind on the primary node. The addresses are automatically unbound when a node becomes secondary. The addresses can therefore be used externally to reliably connect to the HA group primary node.
- -#### **Example Configuration** - -```bash -admin@ncs(config)# hcc enabled -admin@ncs(config)# hcc vip 192.168.123.22 -admin@ncs(config)# hcc vip 2001:db8::10 -admin@ncs(config)# commit -``` - -### Layer-3 BGP - -The purpose of the HCC layer-3 BGP functionality is to operate a BGP daemon on each NSO node and to ensure that routes for the VIP addresses are advertised by the BGP daemon on the primary node only. - -The layer-3 functionality is an optional add-on to the layer-2 functionality. When enabled, the set of BGP neighbors must be configured separately for each NSO node. Each NSO node operates an embedded BGP daemon and maintains connections to peers but only the primary node announces the VIP addresses. - -The layer-3 functionality relies on the layer-2 functionality to assign the virtual IP addresses to one of the host's interfaces. One notable difference in assigning virtual IP addresses when operating in Layer-3 mode is that the virtual IP addresses are assigned to the loopback interface `lo` rather than to a specific physical interface. - -#### **Operational Details** - -HCC operates a [`GoBGP`](https://osrg.github.io/gobgp/) subprocess as an embedded BGP daemon. The BGP daemon is started, configured, and monitored by HCC. The HCC YANG model includes basic BGP configuration data and state data. - -Operational data in the YANG model includes the state of the BGP daemon subprocess and the state of each BGP neighbor connection. The BGP daemon writes log messages directly to NSO where the HCC module extracts updated operational data and then repeats the BGP daemon log messages into the HCC log verbatim. You can find these log messages in the developer log (`devel.log`). - -```bash -admin@ncs# show hcc -NODE BGPD BGPD -ID PID STATUS ADDRESS STATE CONNECTED -------------------------------------------------------------- -london - - 192.168.30.2 - - -paris 827 running 192.168.31.2 ESTABLISHED true -``` - -{% hint style="info" %} -GoBGP must be installed separately. The `gobgp` and `gobgpd` binaries must be found in paths specified by the `$PATH` environment variable. For system install, NSO reads `$PATH` in the `systemd` service file `/etc/systemd/system/ncs.service`. Since tailf-hcc 6.0.2, the path to `gobgp`/`gobgpd` is no longer possible to specify from the configuration data leaf `/hcc/bgp/node/gobgp-bin-dir`. The leaf has been removed from the `tailf-hcc/src/yang/tailf-hcc.yang` module. - -Upgrades: If BGP is enabled and the `gobgp` or `gobgpd` binaries are not found, the tailf-hcc package will fail to load. The user must then install GoBGP and invoke the `packages reload` action or restart NSO with `NCS_RELOAD_PACKAGES=true` in `/etc/ncs/ncs.systemd.conf` and `systemctl restart ncs`. -{% endhint %} - -#### **Configuration** - -The layer-3 BGP functionality is configured as a list of BGP configurations with one list entry per node. Configurations are separate because each NSO node usually has different BGP neighbors with their own IP addresses, authentication parameters, etc. - -The BGP configuration parameters are found under `/hcc:hcc/bgp/node{id}`. - -Per-Node Layer-3 Configuration: - -
ParametersTypeDescription
node-idstringUnique node ID. A reference to /ncs:high-availability/ha-node/id.
enabledbooleanIf set to true, this node uses BGP to announce VIP addresses when in the HA primary state.
asinet:as-numberThe BGP Autonomous System Number for the local BGP daemon.
router-idinet:ip-addressThe router ID for the local BGP daemon.
- -Each NSO node can connect to a different set of BGP neighbors. For each node, the BGP neighbor list configuration parameters are found under `/hcc:hcc/bgp/node{id}/neighbor{address}`. - -Per-Neighbor BGP Configuration: - -
ParametersTypeDescription
addressinet:ip-addressBGP neighbor IP address.
asinet:as-numberBGP neighbor Autonomous System Number.
ttl-minuint8Optional minimum TTL value for BGP packets. When configured, enables BGP Generalized TTL Security Mechanism (GTSM).
passwordstringOptional password to use for BGP authentication with this neighbor.
enabledbooleanIf set to true, then an outgoing BGP connection to this neighbor is established by the HA group primary node.
- -#### **Example** - -```bash -admin@ncs(config)# hcc bgp node paris enabled -admin@ncs(config)# hcc bgp node paris as 64512 -admin@ncs(config)# hcc bgp node paris router-id 192.168.31.99 -admin@ncs(config)# hcc bgp node paris neighbor 192.168.31.2 as 64514 -admin@ncs(config)# ... repeated for each neighbor if more than one ... - ... repeated for each node ... -admin@ncs(config)# commit -``` - -### Layer-3 DNS Update - -The purpose of the HCC layer-3 DNS Update functionality is to notify a DNS server of the IP address change of the active primary NSO server, allowing the DNS server to update the DNS record for the given domain name. - -Geographically redundant NSO setup typically relies on DNS support. To enable this use case, tailf-hcc can dynamically update DNS with the `nsupdate` utility on HA status change notification. - -The DNS server used should support updates through `nsupdate` command (RFC 2136). - -#### Operational Details - -HCC listens on the underlying NSO HA notifications stream. When HCC receives a notification about an NSO node being Primary, it updates the DNS Server with the IP address of the Primary NSO for the given hostname. The HCC YANG model includes basic DNS configuration data and operational status data. - -Operational data in the YANG model includes the result of the latest DNS update operation. - -```bash -admin@ncs# show hcc dns -hcc dns status time 2023-10-20T23:16:33.472522+00:00 -hcc dns status exit-code 0 -``` - -If the DNS Update is unsuccessful, an error message will be populated in operational data, for example: - -```bash -admin@ncs# show hcc dns -hcc dns status time 2023-10-20T23:36:33.372631+00:00 -hcc dns status exit-code 2 -hcc dns status error-message "; Communication with 10.0.0.10#53 failed: timed out" -``` - -{% hint style="info" %} -The DNS Server must be installed and configured separately, and details are provided to HCC as configuration data. The DNS Server must be configured to update the reverse DNS record. -{% endhint %} - -#### Configuration - -The layer-3 DNS Update functionality needs DNS-related information like DNS server IP address, port, zone, etc, and information about NSO nodes involved in HA - node, ip, and location. - -The DNS configuration parameters are found under `/hcc:hcc/dns`. - -Layer-3 DNS Configuration: - -
ParametersTypeDescription
enabledbooleanIf set to true, DNS updates will be enabled.
fqdninet:domain-nameDNS domain-name for the HA primary.
ttluint32Time to live for DNS record, default 86400.
key-filestringSpecifies the file path for nsupdate keyfile.
serverinet:ip-addressDNS Server IP Address.
portuint32DNS Server port, default 53.
zoneinet:hostDNS Zone to update on the server.
timeoutuint32Timeout for nsupdate command, default 300.
- -Each NSO node can be placed in a separate Location/Site/Availability-Zone. This is configured as a list member configuration, with one list entry per node ID. The member list configuration parameters are found under `/hcc:hcc/dns/member{node-id}`. - -
ParameterTypeDescription
node-idstringUnique NSO HA node ID. Valid values are: /high-availability/ha-node when built-in HA is used or /ha-raft/status/member for HA Raft.
ip-addressinet:ip-addressIP where NSO listens for incoming requests to any northbound interfaces.
locationstringName of the Location/Site/Availability-Zone where node is placed.
- -#### Example - -Here is an example configuration for a setup of two dual-stack NSO nodes, node-1 and node-2, that have an IPv4 and an IPv6 address configured. The configuration also sets up an update signing with the specified key. - -```bash -admin@ncs(config)# hcc dns enabled -admin@ncs(config)# hcc dns fqdn example.com -admin@ncs(config)# hcc dns ttl 120 -admin@ncs(config)# hcc dns key-file /home/cisco/DNS-testing/good.key -admin@ncs(config)# hcc dns server 10.0.0.10 -admin@ncs(config)# hcc dns port 53 -admin@ncs(config)# hcc dns zone zone1.nso -admin@ncs(config)# hcc dns member node-1 ip-address [ 10.0.0.20 ::10 ] -admin@ncs(config)# hcc dns member node-1 location SanJose -admin@ncs(config)# hcc dns member node-2 ip-address [ 10.0.0.30 ::20 ] -admin@ncs(config)# hcc dns member node-2 location NewYork -admin@ncs(config)# commit -``` - -### Usage - -This section describes basic deployment scenarios for HCC. Layer-2 mode is demonstrated first and then the layer-3 BGP functionality is configured in addition: - -* [Layer-2 Deployment](high-availability.md#layer-2-deployment) -* [Enabling Layer-3 BGP](high-availability.md#enabling-layer-3-bgp) -* [Enabling Layer-3 DNS](high-availability.md#enabling-layer-3-dns) - -A reference to container-based examples for the layer-2 and layer-3 deployment scenarios described here can be found in the NSO example set under [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc). - -Both scenarios consist of two test nodes: `london` and `paris` with a single IPv4 VIP address. For the layer-2 scenario, the nodes are on the same network. The layer-3 scenario also involves a BGP-enabled `router` node as the `london` and `paris` nodes are on two different networks. - -#### **Layer-2 Deployment** - -The layer-2 operation is configured by simply defining the VIP addresses and enabling HCC. The HCC configuration on both nodes should match, otherwise, the primary node's configuration will overwrite the secondary node configuration when the secondary connects to the primary node. - -Addresses: - -
HostnameAddressRole
paris192.168.23.99Paris service node.
london192.168.23.98London service node.
vip4192.168.23.122NSO primary node IPv4 VIP address.
- -Configuring VIPs: - -```bash -admin@ncs(config)# hcc enabled -admin@ncs(config)# hcc vip 192.168.23.122 -admin@ncs(config)# commit -``` - -Verifying VIP Availability: - -Once enabled, HCC on the HA group primary node will automatically assign the VIP addresses to corresponding Linux network interfaces. - -```bash -root@paris:/var/log/ncs# ip address list -1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1 - link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 - inet 127.0.0.1/8 scope host lo - valid_lft forever preferred_lft forever - inet6 ::1/128 scope host - valid_lft forever preferred_lft forever -2: enp0s3: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 - link/ether 52:54:00:fa:61:99 brd ff:ff:ff:ff:ff:ff - inet 192.168.23.99/24 brd 192.168.23.255 scope global enp0s3 - valid_lft forever preferred_lft forever - inet 192.168.23.122/32 scope global enp0s3 - valid_lft forever preferred_lft forever - inet6 fe80::5054:ff:fefa:6199/64 scope link - valid_lft forever preferred_lft forever -``` - -On the secondary node, HCC will not configure these addresses. - -```bash -root@london:~# ip address list -1: lo: mtu 65536 ... - link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 - inet 127.0.0.1/8 scope host lo - valid_lft forever preferred_lft forever - inet6 ::1/128 scope host - valid_lft forever preferred_lft forever -2: enp0s3: mtu 1500 ... - link/ether 52:54:00:fa:61:98 brd ff:ff:ff:ff:ff:ff - inet 192.168.23.98/24 brd 192.168.23.255 scope global enp0s3 - valid_lft forever preferred_lft forever - inet6 fe80::5054:ff:fefa:6198/64 scope link - valid_lft forever preferred_lft forever -``` - -Layer-2 Example Implementation: - -A reference to a container-based example of the layer-2 scenario can be found in the NSO example set. See the [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc) `README`. - -#### **Enabling Layer-3 BGP** - -Layer-3 operation is configured for each NSO HA group node separately. The HCC configuration on both nodes should match, otherwise, the primary node's configuration will overwrite the configuration on the secondary node. - -Addresses: - -
HostnameAddressASRole
paris192.168.31.9964512Paris node
london192.168.30.9864513London node
router

192.168.30.2

192.168.31.2

64514BGP-enabled router
vip4192.168.23.122Primary node IPv4 VIP address
- -Configuring BGP for Paris Node: - -```bash -admin@ncs(config)# hcc bgp node paris enabled -admin@ncs(config)# hcc bgp node paris as 64512 -admin@ncs(config)# hcc bgp node paris router-id 192.168.31.99 -admin@ncs(config)# hcc bgp node paris neighbor 192.168.31.2 as 64514 -admin@ncs(config)# commit -``` - -Configuring BGP for London Node: - -```bash -admin@ncs(config)# hcc bgp node london enabled -admin@ncs(config)# hcc bgp node london as 64513 -admin@ncs(config)# hcc bgp node london router-id 192.168.30.98 -admin@ncs(config)# hcc bgp node london neighbor 192.168.30.2 as 64514 -admin@ncs(config)# commit -``` - -Check BGP Neighbor Connectivity: - -Check neighbor connectivity on the `paris` primary node. Note that its connection to neighbor 192.168.31.2 (`router`) is `ESTABLISHED`. - -```bash -admin@ncs# show hcc - BGPD BGPD -NODE ID PID STATUS ADDRESS STATE CONNECTED ----------------------------------------------------------------- -london - - 192.168.30.2 - - -paris 2486 running 192.168.31.2 ESTABLISHED true -``` - -Check neighbor connectivity on the `london` secondary node. Note that the primary node also has an `ESTABLISHED` connection to its neighbor 192.168.30.2 (`router`). The primary and secondary nodes both maintain their BGP neighbor connections at all times when BGP is enabled, but only the primary node announces routes for the VIPs. - -```bash -admin@ncs# show hcc - BGPD BGPD -NODE ID PID STATUS ADDRESS STATE CONNECTED ----------------------------------------------------------------- -london 494 running 192.168.30.2 ESTABLISHED true -paris - - 192.168.31.2 - - -``` - -Check Advertised BGP Routes Neighbors: - -Check the BGP routes received by the `router`. - -```bash -admin@ncs# show ip bgp -... -Network Next Hop Metric LocPrf Weight Path -*> 192.168.23.122/32 - 192.168.31.99 0 64513 ? -``` - -The VIP subnet is routed to the `paris` host, which is the primary node. - -Layer-3 BGP Example Implementation: - -A reference to a container-based example of the combined layer-2 and layer-3 BGP scenario can be found in the NSO example set. See the [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc) `README`. - -#### **Enabling Layer-3 DNS** - -If enabled prior to the HA being established, HCC will update the DNS server with the IP address of the Primary node once a primary is selected. - -If an HA is already operational, and Layer-3 DNS is enabled and configured afterward, HCC will not update the DNS server automatically. An automatic DNS server update will only happen if a HA switchover happens. HCC exposes an update action to manually trigger an update to the DNS server with the IP address of the primary node. - -DNS Update Action: - -The user can explicitly update DNS from the specific NSO node by running the update action. - -```bash -admin@ncs# hcc dns update -``` - -Check the result of invoking the DNS update utility using the operational data in `/hcc/dns`: - -```bash -admin@ncs# show hcc dns -hcc dns status time 2023-10-10T20:47:31.733661+00:00 -hcc dns status exit-code 0 -hcc dns status error-message "" -``` - -One way to verify DNS server updates is through the `nslookup` program. However, be mindful of the DNS caching mechanism, which may cache the old value for the amount of time controlled by the TTL setting. - -```bash -cisco@node-2:~$ nslookup example.com -Server: 10.0.0.10 -Address: 10.0.0.10#53 - -Name: example.com -Address: 10.0.0.20 -Name: example.com -Address: ::10 -``` - -DNS get-node-location Action: - -/hcc/dns/member holds the information about all members involved in HA. The `get-node-location` action provides information on the location of an NSO node. - -```bash -admin@ncs(config)# hcc dns get-node-location -location SanJose -``` - -### Data Model - -The HCC data model can be found in the HCC package (`tailf-hcc.yang`). - -## Setup with an External Load Balancer - -As an alternative to the HCC package, NSO built-in HA, either rule-based or HA Raft, can also be used in conjunction with a load balancer device in a reverse proxy configuration. Instead of managing the virtual IP address directly as HCC does, this setup relies on an external load balancer to route traffic to the currently active primary node. - -

Load Balancer Routes Connections to the Appropriate NSO Node

- -The load balancer uses HTTP health checks to determine which node is currently the active primary. The example, found in the [examples.ncs/high-availability/load-balancer](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/load-balancer) directory uses HTTP status codes on the health check endpoint to easily distinguish whether the node is currently primary or not. - -In the example, freely available HAProxy software is used as a load balancer to demonstrate the functionality. It is configured to steer connections on localhost to either of the TCP port 2024 (SSH CLI) and TCP port 8080 (web UI and RESTCONF) to the active node in a 2-node HA cluster. The HAProxy software is required if you wish to run this example yourself. - -

Load Balancer Uses Health Checks to Determine the Currently Active Primary Node

- -You can start all the components in the example by running the `make build start` command. At the beginning, the first node `n1` is the active primary. Connecting to the localhost port 2024 will establish a connection to this node: - -```bash -$ make build start -Setting up run directory for nso-node1 - ... make output omitted ... -Waiting for n2 to connect: . -$ ssh -p 2024 admin@localhost -admin@localhost's password: admin - -admin connected from 127.0.0.1 using ssh on localhost -admin@n1> switch cli -admin@n1# show high-availability -high-availability enabled -high-availability status mode primary -high-availability status current-id n1 -high-availability status assigned-role primary -high-availability status read-only-mode false -ID ADDRESS ---------------- -n2 127.0.0.1 -``` - -Then, you can disable the high availability subsystem on `n1` to simulate a node failure. - -```bash -admin@n1# high-availability disable -result NSO Built-in HA disabled -admin@n1# exit -Connection to localhost closed. -``` - -Disconnect and wait a few seconds for the built-in HA to perform the failover to node `n2`. The time depends on the `high-availability/settings/reconnect-interval` and is set quite aggressively in this example to make the failover in about 6 seconds. Reconnect with the SSH client and observe the connection is now made to the fail-over node which has become the active primary: - -```bash -$ ssh -p 2024 admin@localhost -admin@localhost's password: admin - -admin connected from 127.0.0.1 using ssh on localhost -admin@n2> switch cli -admin@n2# show high-availability -high-availability enabled -high-availability status mode primary -high-availability status current-id n2 -high-availability status assigned-role primary -high-availability status read-only-mode false -``` - -Finally, shut down the example with the `make stop clean` command. - -## NB Listens to Addresses on HA Primary for Load Balancers - -NSO can be configured for the HA primary to listen on additional ports for the northbound interfaces NETCONF, RESTCONF, the web server (including JSON-RPC), and the CLI over SSH. Once a different node transitions to role primary the configured listen addresses are brought up on that node instead. - -When the following configuration is added to `ncs.conf`, then the primary HA node will listen(2) and bind(2) port 1830 on the wildcard IPv4 and IPv6 addresses. - -```xml - - - - true - 0.0.0.0 - 830 - - 0.0.0.0 - 1830 - - - :: - 1830 - - - - -``` - -A similar configuration can be added for other NB interfaces, see the ha-primary-listen list under `/ncs-config/{restconf,webui,cli}`. - -## HA Framework Requirements - -If an external HAFW is used, NSO only replicates the CDB data. NSO must be told by the HAFW which node should be primary and which nodes should be secondaries. - -The HA framework must also detect when nodes fail and instruct NSO accordingly. If the primary node fails, the HAFW must elect one of the remaining secondaries and appoint it the new primary. The remaining secondaries must also be informed by the HAFW about the new primary situation. - -### Mode of Operation - -NSO must be instructed through the `ncs.conf` configuration file that it should run in HA mode. The following configuration snippet enables HA mode: - -```xml - - true - 0.0.0.0 - 4570 - - :: - 4569 - - PT20S - -``` - -Make sure to restart the `ncs` process for the changes to take effect. - -The IP address and the port above indicate which IP and which port should be used for the communication between the HA nodes. `extra-listen` is an optional list of `ip:port` pairs that a HA primary also listens to for secondary connections. For IPv6 addresses, the syntax `[ip]:port` may be used. If the `:port` is omitted, `port` is used. The `tick-timeout` is a duration indicating how often each secondary must send a tick message to the primary indicating liveness. If the primary has not received a tick from a secondary within 3 times the configured tick time, the secondary is considered to be dead. Similarly, the primary sends tick messages to all the secondaries. If a secondary has not received any tick messages from the primary within the 3 times the timeout, the secondary will consider the primary dead and report accordingly. - -A HA node can be in one of three states: `NONE`, `SECONDARY` or `PRIMARY`. Initially, a node is in the `NONE` state. This implies that the node will read its configuration from CDB, stored locally on file. Once the HA framework has decided whether the node should be a secondary or a primary the HAFW must invoke either the methods `Ha.beSecondary(primary)` or `Ha.bePrimary()` - -When an NSO HA node starts, it always starts up in mode `NONE`. At this point, there are no other nodes connected. Each NSO node reads its configuration data from the locally stored CDB and applications on or off the node may connect to NSO and read the data they need. Although write operations are allowed in the `NONE` state it is highly discouraged to initiate southbound communication unless necessary. A node in `NONE` state should only be used to configure NSO itself or to do maintenance such as upgrades. When in `NONE` state, some features are disabled, including but not limited to: - -* commit queue -* NSO scheduler -* nano-service side effect queue - -This is to avoid situations where multiple NSO nodes are trying to perform the same southbound operation simultaneously. - -At some point, the HAFW will command some nodes to become secondary nodes of a named primary node. When this happens, each secondary node tracks changes and (logically or physically) copies all the data from the primary. Previous data at the secondary node is overwritten. - -Note that the HAFW, by using NSO's start phases, can make sure that NSO does not start its northbound interfaces (NETCONF, CLI, ...) until the HAFW has decided what type of node it is. Furthermore once a node has been set to the `SECONDARY` state, it is not possible to initiate new write transactions towards the node. It is thus never possible for an agent to write directly into a secondary node. Once a node is returned either to the `NONE` state or to the `PRIMARY` state, write transactions can once again be initiated towards the node. - -The HAFW may command a secondary node to become primary at any time. The secondary node already has up-to-date data, so it simply stops receiving updates from the previous primary. Presumably, the HAFW also commands the primary node to become a secondary node or takes it down, or handles the situation somehow. If it has crashed, the HAFW tells the secondary to become primary, restarts the necessary services on the previous primary node, and gives it an appropriate role, such as secondary. This is outside the scope of NSO. - -Each of the primary and secondary nodes has the same set of all callpoints and validation points locally on each node. The start sequence has to make sure the corresponding daemons are started before the HAFW starts directing secondary nodes to the primary, and before replication starts. The associated callbacks will however only be executed at the primary. If e.g. the validation executing at the primary needs to read data that is not stored in the configuration and only available on another node, the validation code must perform any needed RPC calls. - -If the order from the HAFW is to become primary, the node will start to listen for incoming secondaries at the `ip:port` configured under `/ncs-config/ha`. The secondaries TCP connect to the primary and this socket is used by NSO to distribute the replicated data. - -If the order is to be a secondary, the node will contact the primary and possibly copy the entire configuration from the primary. This copy is not performed if the primary and secondary decide that they have the same version of the CDB database loaded, in which case nothing needs to be copied. This mechanism is implemented by use of a unique token, the `transaction id` - it contains the node id of the node that generated it and a time stamp, but is effectively "opaque". - -This transaction ID is generated by the cluster primary each time a configuration change is committed, and all nodes write the same transaction ID into their copy of the committed configuration. If the primary dies and one of the remaining secondaries is appointed the new primary, the other secondaries must be told to connect to the new primary. They will compare their last transaction ID to the one from the newly appointed primary. If they are the same, no CDB copy occurs. This will be the case unless a configuration change has sneaked in since both the new primary and the remaining secondaries will still have the last transaction ID generated by the old primary - the new primary will not generate a new transaction ID until a new configuration change is committed. The same mechanism works if a secondary node is simply restarted. No cluster reconfiguration will lead to a CDB copy unless the configuration has been changed in between. - -Northbound agents should run on the primary, an agent can't commit write operations at a secondary node. - -When an agent commits its CDB data, CDB will stream the committed data out to all registered secondaries. If a secondary dies during the commit, nothing will happen, the commit will succeed anyway. When and if the secondary reconnects to the cluster, the secondary will have to copy the entire configuration. All data on the HA sockets between NSO nodes only go in the direction from the primary to the secondaries. A secondary that isn't reading its data will eventually lead to a situation with full TCP buffers at the primary. In principle, it is the responsibility of HAFW to discover this situation and notify the primary NSO about the hanging secondary. However, if 3 times the tick timeout is exceeded, NSO will itself consider the node dead and notify the HAFW. The default value for tick timeout is 20 seconds. - -The primary node holds the active copy of the entire configuration data in CDB. All configuration data has to be stored in CDB for replication to work. At a secondary node, any request to read will be serviced while write requests will be refused. Thus, the CDB subscription code works the same regardless of whether the CDB client is running at the primary or at any of the secondaries. Once a secondary has received the updates associated to a commit at the primary, all CDB subscribers at the secondary will be duly notified about any changes using the normal CDB subscription mechanism. - -If the system has been set up to subscribe for NETCONF notifications, the secondaries will have all subscriptions as configured in the system, but the subscription will be idle. All NETCONF notifications are handled by the primary, and once the notifications get written into stable storage (CDB) at the primary, the list of received notifications will be replicated to all secondaries. - -## Security Aspects - -We specify in `ncs.conf` which IP address the primary should bind for incoming secondaries. If we choose the default value `0.0.0.0` it is the responsibility of the application to ensure that connection requests only arrive from acceptable trusted sources through some means of firewalling. - -A cluster is also protected by a token, a secret string only known to the application. The `Ha.connect()` method must be given the token. A secondary node that connects to a primary node negotiates with the primary using a CHAP-2-like protocol, thus both the primary and the secondary are ensured that the other end has the same token without ever revealing their own token. The token is never sent in clear text over the network. This mechanism ensures that a connection from an NSO secondary to a primary can only succeed if they both have the same token. - -It is indeed possible to store the token itself in CDB, thus an application can initially read the token from the local CDB data, and then use that token in . the constructor for the `Ha` class. In this case, it may very well be a good idea to have the token stored in CDB be of type tailf:aes-256-cfb-128-encrypted-string. - -If the actual CDB data that is sent on the wire between cluster nodes is sensitive, and the network is untrusted, the recommendation is to use IPSec between the nodes. An alternative option is to decide exactly which configuration data is sensitive and then use the tailf:aes-256-cfb-128-encrypted-string type for that data. If the configuration data is of type tailf:aes-256-cfb-128-encrypted-string the encrypted data will be sent on the wire in update messages from the primary to the secondaries. - -## API - -There are two APIs used by the HA framework to control the replication aspects of NSO. First, there exists a synchronous API used to tell NSO what to do, secondly, the application may create a notifications socket and subscribe to HA-related events where NSO notifies the application on certain HA-related events such as the loss of the primary, etc. The HA-related notifications sent by NSO are crucial to how to program the HA framework. - -The HA-related classes reside in the `com.tailf.ha` package. See Javadocs for reference. The HA notifications-related classes reside in the `com.tailf.notif` package, See Javadocs for reference. - -## Ticks - -The configuration parameter `/ncs-cfg/ha/tick-timeout` is by default set to 20 seconds. This means that every 20 seconds each secondary will send a tick message on the socket leading to the primary. Similarly, the primary will send a tick message every 20 seconds on every secondary socket. - -This aliveness detection mechanism is necessary for NSO. If a socket gets closed all is well, NSO will clean up and notify the application accordingly using the notifications API. However, if a remote node freezes, the socket will not get properly closed at the other end. NSO will distribute update data from the primary to the secondaries. If a remote node is not reading the data, TCP buffer will get full and NSO will have to start to buffer the data. NSO will buffer data for at most `tickTime` times 3 time units. If a `tick` has not been received from a remote node within that time, the node will be considered dead. NSO will report accordingly over the notifications socket and either remove the hanging secondary or, if it is a secondary that loses contact with the primary, go into the initial `NONE` state. - -If the HAFW can be really trusted, it is possible to set this timeout to `PT0S`, i.e zero, in which case the entire dead-node-detection mechanism in NSO is disabled. - -## Relay Secondaries - -The normal setup of an NSO HA cluster is to have all secondaries connected directly to the primary. This is a configuration that is both conceptually simple and reasonably straightforward to manage for the HAFW. In some scenarios, in particular a cluster with multiple secondaries at a location that is network-wise distant from the primary, it can however be sub-optimal, since the replicated data will be sent to each remote secondary individually over a potentially low-bandwidth network connection. - -To make this case more efficient, we can instruct a secondary to be a relay for other secondaries, by invoking the `Ha.beRelay()` method. This will make the secondary start listening on the IP address and port configured for HA in `ncs.conf`, and handle connections from other secondaries in the same manner as the cluster primary does. The initial CDB copy (if needed) to a new secondary will be done from the relay secondary, and when the relay secondary receives CDB data for replication from its primary, it will distribute the data to all its connected secondaries in addition to updating its own CDB copy. - -To instruct a node to become a secondary connected to a relay secondary, we use the `Ha.beSecondary()` method as usual, but pass the node information for the relay secondary instead of the node information for the primary. I.e. the "sub-secondary" will in effect consider the relay secondary as its primary. To instruct a relay secondary to stop being a relay, we can invoke the `Ha.beSecondary()` method with the same parameters as in the original call. This is a no-op for a "normal" secondary, but it will cause a relay secondary to stop listening for secondary connections, and disconnect any already connected "sub-secondaries". - -This setup requires special consideration by the HAFW. Instead of just telling each secondary to connect to the primary independently, it must set up the secondaries that are intended to be relays, and tell them to become relays, before telling the "sub-secondaries" to connect to the relay secondaries. Consider the case of a primary M and a secondary S0 in one location, and two secondaries S1 and S2 in a remote location, where we want S1 to act as relay for S2. The setup of the cluster then needs to follow this procedure: - -1. Tell M to be primary. -2. Tell S0 and S1 to be secondary with M as primary. -3. Tell S1 to be relay. -4. Tell S2 to be secondary with S1 as primary. - -Conversely, the handling of network outages and node failures must also take the relay secondary setup into account. For example, if a relay secondary loses contact with its primary, it will transition to the `NONE` state just like any other secondary, and it will then disconnect its sub-secondaries which will cause those to transition to `NONE` too, since they lost contact with "their" primary. Or if a relay secondary dies in a way that is detected by its sub-secondaries, they will also transition to `NONE`. Thus in the example above, S1 and S2 needs to be handled differently. E.g. if S2 dies, the HAFW probably won't take any action, but if S1 dies, it makes sense to instruct S2 to be a secondary of M instead (and when S1 comes back, perhaps tell S2 to be a relay and S1 to be a secondary of S2). - -Besides the use of `Ha.beRelay()`, the API is mostly unchanged when using relay secondaries. The HA event notifications reporting the arrival or the death of a secondary are still generated only by the "real" cluster primary. If the `Ha.HaStatus()` method is used towards a relay secondary, it will report the node state as `SECONDARY_RELAY` rather than just `SECONDARY`, and the array of nodes will have its primary as the first element (same as for a "normal" secondary), followed by its "sub-secondaries" (if any). - -## CDB Replication - -When HA is enabled in `ncs.conf`, CDB automatically replicates data written on the primary to the connected secondary nodes. Replication is done on a per-transaction basis to all the secondaries in parallel and is synchronous. When NSO is in secondary mode the northbound APIs are in read-only mode, that is the configuration can not be changed on a secondary other than through replication updates from the primary. It is still possible to read from for example NETCONF or CLI (if they are enabled) on a secondary. CDB subscriptions work as usual. When NSO is in the `NONE` state CDB is unlocked and it behaves as when NSO is not in HA mode at all. - -Unlike configuration data, operational data is replicated only if it is defined as persistent in the data model (using the `tailf:persistent` extension). diff --git a/administration/management/ned-administration.md b/administration/management/ned-administration.md deleted file mode 100644 index 34ca6f4b..00000000 --- a/administration/management/ned-administration.md +++ /dev/null @@ -1,963 +0,0 @@ ---- -description: Learn about Cisco-provided NEDs and how to manage them. ---- - -# NED Administration - -This section provides necessary information on Network Element Driver (NED) administration with a focus on Cisco-provided NEDs. If you're planning to use NEDs not provided by Cisco, refer to the [NED Development](../../development/advanced-development/developing-neds/) to build your own NED packages. - -## NED Introduction - -NED represents a key NSO component that makes it possible for the NSO core system to communicate southbound with network devices in most deployments. NSO has a built-in client that can be used to communicate southbound with NETCONF-enabled devices. Many network devices are, however, not NETCONF-enabled, and there exist a wide variety of methods and protocols for configuring network devices, ranging from simple CLI to HTTP/REST-enabled devices. For such cases, it is necessary to use a NED to allow NSO communicate southbound with the network device. - -Even for NETCONF-enabled devices, it is possible that the NSO's built-in NETCONF client cannot be used, for instance, if the devices do not strictly follow the specification for the NETCONF protocol. In such cases, one must also use a NED to seamlessly communicate with the device. See [Managing Cisco-provided third Party YANG NEDs](ned-administration.md#sec.managing_thirdparty_neds) for more information on third-party YANG NEDs. - -### NED Contents and Capabilities - -It's important to understand the functionality of a NED and the capabilities it offers — as well as those it does not. The following summarizes what a NED contains and what it doesn't. - -#### **What a NED Provides** - -
- -YANG Data Model - -The NED provides a YANG data model of the device to NSO and services, enabling standardized configuration management. This applies only to NEDs where Cisco creates and maintains the device data model—commonly referred to as classic NEDs, which includes both the CLI-based and Generic NEDs—and excludes third-party YANG (3PY) NEDs, where the model is provided externally.\ -\ -Note that for classic NEDs, the device model is typically implemented as a superset, covering multiple versions or variants of a given device type. This approach allows a single NED package to support a broad range of software versions or hardware flavors. The benefit is simplified deployment and upgrade handling across similar devices. However, a side effect is that certain parts of the model may not apply to the specific device instance in use. - -
- -
- -Data Translation - -The NED is responsible for transforming outbound data from NSO's internal format into a format understood by the device — whether that format is vendor-specific (e.g., CLI, REST, SOAP) or standards-based (e.g., NETCONF, RESTCONF, gNMI). It also handles the reverse transformation for inbound data from the device back into NSO's format. - -
- -NSO ensures all data modifications occur within a single transaction for consistency and guarantees a transaction is either completely successful or fails, maintaining data integrity. - -#### **What a NED Does not Provide** - -
- -A Data Model of the Entire Set in the Data - -For Classic NEDs, NED development is use-case driven. As a result, a NED, in most cases, does not contain the complete data model of a device. Providing a 100% complete YANG model for a device is not a goal and is not in the scope of NED development. It does not make sense to invest resources into modeling data which is not needed to support the desired use cases. If a NED does not cover a needed use case, please submit an enhancement request via your support channel. For third party NEDs, the models come from third party sources not controlled by Cisco. - -
- -
- -An Exact Copy of the Syntax in the Device CLI - -NED development focuses on representing device data for NSO. As a side effect for CLI NEDs, the NSO CLI will get similar behavior as the device CLI, however, in most situations, this will not be perfect and is not the goal of the NED. - -
- -
- -Fine-grained Validation of Data (Classic NEDs Only) - -In classic NEDs, adding strict validations in the YANG model (e.g., `mandatory`, `when`, `must`, `range`, `min`, `max`, etc.) can lead to inflexible models. These constraints are interpreted and enforced by NSO at runtime, not the device. Since such validations often need to be updated as devices evolve across versions, NSO's policy is to keep the models relaxed by minimizing the use of these validation constructs. This allows for greater flexibility and forward compatibility. - -
- -
- -Convenience Macros in the Device CLI (Only Discrete Data Leaves are Supported) - -Some devices have macro-style functionality in the CLI and users may find it annoying that these are not available in NEDs. The convenience macros have proven very dynamic in the parameters they change, causing frequent out-of-sync situations, but these are generally not available in the NED. - -
- -
- -Dynamic Configuration in Devices (Only Data in a Transaction May Change) - -Cisco NEDs do not model device-generated or dynamic configuration, as such behavior varies between device versions and is difficult to standardize. Only configuration explicitly included in a transaction is managed by NSO. If needed, service logic can insert expected dynamic elements during provisioning. - -
- -
- -Auto-correction of Parameters with Multiple Syntaxes (i.e., Use Canonical Form) - -The NED does not allow the same value for a parameter to have a different name (e.g., `true` vs. `yes`). The canonical name displayed in `show-running-config` or similar is used. - -
- -
- -Handling Out-of-band Changes (Model as Operational Data) - -Leaves that have out-of-band changes will cause NSO and the device to become out- of-sync, and should be made "config false", or not be part of the model at all. Similarly, actions that cause out-of-band changes are not supported. - -
- -
- -Splitting a Single Transaction into Several Sub-transactions - -For devices that support the transaction paradigm, the NED will never split an NSO transaction in two or more device transactions. The service must handle this by doing multiple NSO transactions. - -
- -
- -Backporting of Fixes to Old NED Releases (i.e., Trunk based Development is Used) - -All NEDs use trunk-based development, i.e., new NED releases are created from the tip of a single branch, develop. New features and fixes are thus delivered to the stakeholders in the latest NED release, not by backporting an old release. - -
- -## Types of NED Packages - -A NED package is a package that NSO uses to manage a particular type of device. A NED is a piece of code that enables communication with a particular type of managed device. You add NEDs to NSO as a special kind of package, called NED packages. - -A NED package must provide a device YANG model as well as define means (protocol) to communicate with the device. The latter can either leverage the NSO built-in NETCONF and SNMP support or use a custom implementation. When a package provides custom protocol implementation, typically written in Java, it is called a CLI NED or a Generic NED. - -Cisco provides and supports a number of such NEDs. With these Cisco-provided NEDs, a major category are CLI NEDs which communicate with a device through its CLI instead of a dedicated API. - -

NED Package Types

- -### NED Types Summary Table - -
NED CategoryPurposeProviderYANG Model ProviderYANG Models Included?Device InterfaceProtocols SupportedKey Characteristics
CLI NED*Designed for devices with a CLI-based interface. The NED parses CLI commands and translates data to/from YANG.CiscoCisco NSO NED TeamYesCLI (Command Line Interface)SSH, Telnet
  • Mimics CLI command hierarchy
  • Turbo parser for CLI parsing
  • Transform engines for data conversion
  • Targets devices using CLI as config interface
Generic NED - Cisco YANG Models*Built for API-based devices (e.g., REST, SOAP, TL1), using custom parsers and data transformation logic maintained by Cisco.CiscoCisco NSO NED TeamYesNon-CLI (API-based)REST, TL1, CORBA, SOAP, RESTCONF, gNMI, NETCONF
  • Model-driven devices
  • YANG models mimic proprietary protocol messages
  • JSON/XML transformers
  • Custom protocol implementations
Third-party YANG NEDCisco-supplied generic NED packages that do not include any device models.CiscoThird-party Vendors/Organizations (IETF, IEEE, ONF, OpenConfig)No - Must be downloaded separatelyModel-driven protocolsNETCONF, RESTCONF, gNMI
  • Delivered without YANG models
  • Requires download and rebuild process
  • Includes recipes for YANG/device fixes
  • Legal restrictions prevent Cisco redistribution
- -\*Also referred to as Classic NED. - -### CLI NED - -This NED category is targeted at devices that use CLI as a configuration interface. Cisco-provided CLI NEDs are available for various network devices from different vendors. Many different CLI syntaxes are supported. - -The driver element in a CLI NED implemented by the Cisco NSO NED team typically consists of the following three parts: - -* The protocol client, responsible for connecting to and interacting with the device. The protocols supported are SSH and Telnet. -* A fast and versatile CLI parser (+ emitter), usually referred to as the turbo parser. -* Various transform engines capable of converting data between NSO and device formats. - -The YANG models in a CLI NED are developed and maintained by the Cisco NSO NED team. Usually, the models for a CLI NED are structured to mimic the CLI command hierarchy on the device. - -

CLI NED

- -### Generic NED - -A Generic NED is typically used to communicate with non-CLI devices, such as devices using protocols like REST, TL1, Corba, SOAP, RESTCONF, or gNMI as a configuration interface. Even NETCONF-enabled devices in many cases require a generic NED to function properly with NSO. - -The driver element in a Generic NED implemented by the Cisco NED team typically consists of the following parts: - -* The protocol client, responsible for interacting with the device. -* Various transform engines capable of converting data between NSO and the device formats, usually JSON and/or XML transformers. - -There are two types of Generic NEDs maintained by the Cisco NSO NED team: - -* NEDs with Cisco-owned YANG models. These NEDs have models developed and maintained by the Cisco NSO NED team. -* NEDs targeted at YANG models from third-party vendors, also known as, third-party YANG NEDs. - -### **Generic Cisco-provided NEDs with Cisco-owned YANG Models** - -Generic NEDs belonging to the first category typically handle devices that are not model-driven. For instance, devices using proprietary protocols based on REST, SOAP, Corba, etc. The YANG models for such NEDs are usually structured to mimic the messages used by the proprietary protocol of the device. - -

Generic NED

- -### **Third-party YANG NEDs** - -As the name implies, this NED category is used for cases where the device YANG models are not implemented, maintained, or owned by the Cisco NSO NED team. Instead, the YANG models are typically provided by the device vendor itself, or by organizations like IETF, IEEE, ONF, or OpenConfig. - -This category of NEDs has some special characteristics that set them apart from all other NEDs developed by the Cisco NSO NED team: - -* Targeted for devices supporting model-driven protocols like NETCONF, RESTCONF, and gNMI. -* Delivered from the software.cisco.com portal without any device YANG models included. There are several reasons for this, such as legal restrictions that prevent Cisco from re-distributing YANG models from other vendors, or the availability of several different version bundles for open-source YANG, like OpenConfig. The version used by the NED must match the version used by the targeted device. -* The NEDs can be bundled with various fixes to solve shortcomings in the YANG models, the download sources, and/or in the device. These fixes are referred to as recipes. - -

Third-Party YANG NEDs

- -Since the third-party NEDs are delivered without any device YANG models, there are additional steps required to make this category of NEDs operational: - -1. The device models need to be downloaded and copied into the NED package source tree. This can be done by using a special (optional) downloader tool bundled with each third-party YANG NED, or in any custom way. -2. The NED must be rebuilt with the downloaded YANG models. - -This procedure is thoroughly described in [Managing Cisco-provided third-Party YANG NEDs](ned-administration.md#sec.managing_thirdparty_neds). - -#### **Recipes** - -A third-party YANG NED can be bundled with up to three types of recipe modules. These recipes are used by the NED to solve various types of issues related to: - -* The source of the YANG files. -* The YANG files. -* The device itself. - -The recipes represent the characteristics and the real value of a third-party YANG NED. Recipes are typically adapted for a certain bundle of YANG models and/or certain device types. This is why there exist many different third-party YANG NEDs, each one adapted for a specific protocol, a specific model package, and/or a specific device. - -{% hint style="info" %} -The NSO NED team does not provide any super third-party YANG NEDs, for instance, a super RESTCONF NED that can be used with any models and any device. -{% endhint %} - -**Third-party YANG NED Recipe Types** - -
Recipe TypePurposeDescription
Download Recipes (DR)YANG Model Sourcing
  • Presets for downloader tool
  • Define download sources (device, Git repos, archives)
  • Limit scope of YANG files to download
  • Multiple profiles per NED
YANG Recipes (YR)YANG File Fixes
  • Patch downloaded YANG files before compilation
  • Fix compilation errors and YANG construct issues
  • Applied automatically during make process
Runtime Recipes (RR)Device Behavior Fixes
  • Handle device runtime deviations
  • Fix protocol implementation issues
  • Clean up "dirty" configuration dumps
  • Handle device aliasing issues
  • Configurable via runtime profiles
- -**Download Recipes (or Download Profiles)** - -When downloading the YANG files, it is first of all important to know which source to use. In some cases, the source is the device itself. For instance, if the device is enabled for NETCONF and sometimes for RESTCONF (in rare cases). - -In other cases, the device does not support model download. This applies to all gNMI-enabled devices and most RESTCONF devices too. In this case, the source can be a public Git repository or an archive file provided by the device vendor. - -Another important question is what YANG models and what versions to download. To make this task easier, third-party NEDs can be bundled with the download recipes (also known as download profiles). These are presets to be used with the downloader tool bundled with the NED. There can be several profiles, each representing a preset that has been verified to work by the Cisco NSO NED team. A profile can point out a certain source to download from. It can also limit the scope of the download so that only certain YANG files are selected. - -**YANG Recipes (YR)** - -Third-party YANG files can often contain various types of errors, ranging from real bugs that cause compilation errors to certain YANG constructs that are known to cause runtime issues in NSO. To ensure that the files can be built correctly, the third-party NEDs can be bundled with YANG recipes. These recipes patch the downloaded YANG files before they are built by the NSO compiler. This procedure is performed automatically by the `make` system when the NED is rebuilt after downloading the device YANG files. For more information, refer to the procedure related to rebuilding the NED with a unique NED ID in NED READMEs. - -In some cases, YANG recipes are also necessary when a device does not fully conform to the behavior described by its advertised YANG models. This often happens when the device is more permissive than the model suggests—for example, allowing optional parameters that the model marks as mandatory, or omitting data that is expected. Such mismatches can lead to runtime issues in NSO, such as `sync-from` failures or commit errors. YANG recipes allow patching the models to reflect the actual device behavior more accurately. - -**Runtime Recipes (RR)** - -Many devices enabled for NETCONF, RESTCONF, or gNMI sometimes deviate in their runtime behavior. This can make it impossible to interact properly with NSO. These deviations can be on any level in the runtime behavior, such as: - -* The configuration protocol is not properly implemented, i.e., the device lacks support for mandatory parts of, for instance, the RESTCONF RFC. -* The device returns "dirty" configuration dumps, for instance, JSON or XML containing invalid elements. -* Special quirks are required when applying new configuration on a device. May also require additional transforms of the payload before it is relayed by the NED. -* The device has aliasing issues, possibly caused by overlapping YANG models. If leaf X in model A is modified, the device will automatically modify leaf Y in model B as well. While this can be a cause of deviation, note that resolving aliasing issues through runtime recipes is generally avoided by NSO, as it is typically considered a modeling error. - -A third-party YANG NED can be bundled with runtime recipes to solve these kinds of issues, if necessary. How this is implemented varies from NED to NED. In some cases, a NED has a fixed set of recipes that are always used. Alternatively, a NED can support several different recipes, which can be configured through a NED setting, referred to as a runtime profile. For example, a multi-vendor third-party YANG NED might have one runtime profile for each device type supported: - -```bash -admin@ncs(config)# devices device dev-1 ned-settings -onf-tapi_rc restconf profile vendor-xyz -``` - -### NED Settings - -NED settings are YANG models augmented as configurations in NSO and control the behavior of the NED. These settings are augmented under: - -* `/devices/global-settings/ned-settings` -* `/devices/profiles/ned-settings` -* `/devices/device/ned-settings` - -Most NEDs are instrumented with a large number of NED settings that can be used to customize the device instance configured in NSO. The README file in the respective NED contains more information on these. - -## Purpose of NED ID - -Each managed device in NSO has a device type that informs NSO how to communicate with the device. When managing NEDs, the device type is either `cli` or `generic`. The other two device types, `netconf` and `snmp`, are used in NETCONF and SNMP packages and are further described in this guide. - -In addition, a special NED ID identifier is needed. Simply put, this identifier is a handle in NSO pointing to the NED package. NSO uses the identifier when it is about to invoke the driver in a NED package. The identifier ensures that the driver of the correct NED package is called for a given device instance. For more information on how to set up a new device instance, see [Configuring a device with the new Cisco-provided NED](ned-administration.md#sec.config_device.with.ciscoid). - -Each NED package has a NED ID, which is mandatory. The NED ID is a simple string that can have any format. For NEDs developed by the Cisco NSO NED team, the NED ID is formatted as `--.`. - -**Examples** - -* `onf-tapi_rc-gen-2.0` -* `cisco-iosxr-cli-7.43` - -The NED ID for a certain NED package stays the same from one version to another, as long as no backward incompatible changes have been introduced to the YANG models. Upgrading a NED from one version to another, where the NED ID is the same, is simple as it only requires replacing the old NED package with the new one in NSO and then reloading all packages. For third-party (3PY) NEDs, such as the `onf-tapi_rc` NED, the situation differs slightly. Since the YANG models originate from external sources, the NED team does not control their evolution or guarantee backward compatibility between revisions. As a result, it is the responsibility of the end user to determine whether changes in the third-party YANG models are backward compatible and to choose an appropriate version and NED ID when rebuilding the NED. Unlike classic NEDs, upgrading a 3PY NED may therefore require more careful validation and potentially a change in NED ID to reflect incompatibilities. - -Upgrading a NED package from one version to another, where the NED ID is not the same (typically indicated by a change of major or minor number in the NED version), requires additional steps. The new NED package first needs to be installed side-by-side with the old one. Then, a NED migration needs to be performed. This procedure is thoroughly described in [NED Migration](ned-administration.md#sec.ned_migration). - -The Cisco NSO NED team ensures that our CLI NEDs, as well as Generic NEDs with Cisco-owned models, have version numbers and NED ID that indicate any possible backward incompatible YANG model changes. When a NED with such an incompatible change is released, the minor digit in the version is always incremented. The case is a bit different for our third-party YANG NEDs since it is up to the end user to select the NED ID to be used. This is further described in [Managing Cisco-provided third-Party YANG NEDs](ned-administration.md#sec.managing_thirdparty_neds). - -### NED Versioning Scheme (Classic NEDs Only) - -{% hint style="warning" %} -Not applicable to Cisco third-party NEDs. -{% endhint %} - -A NED is assigned a version number consisting of a sequence of numbers separated by dots. The first two numbers represent the major and minor version, and the third number represents the maintenance version. - -For example, the number 5.8.1 indicates a maintenance release (1) for the minor release 5.8. Incompatible YANG model changes require either the major or minor version number to be changed. This means that any version within the 5.8.x series is backward compatible with the previous versions. - -When a newer maintenance release with the same major/minor version replaces a NED release, NSO can perform a simple data model upgrade to handle stored instance data in the CDB (Configuration Database). This type of upgrade does not pose a risk of data loss. - -However, when a NED is replaced by a new major/minor release, it becomes a NED migration. These migrations are complex because the YANG model changes can potentially result in the loss of instance data if not handled correctly. - -

NED Version Scheme

- -## Installing a NED in NSO - -This section describes the NED installation in NSO for Local and System installs. - -{% tabs %} -{% tab title="NED Installation on Local Install" %} -{% hint style="info" %} -This procedure below broadly outlines the steps needed to install a NED package on a [Local Install](../installation-and-deployment/local-install.md). For most up-to-date and specific installation instructions, consult the `README.md` supplied with the NED. -{% endhint %} - -General instructions to install a NED package: - -1. Download the latest production-grade version of the NED from [software.cisco.com](https://software.cisco.com) using the URLs provided on your NED license certificates. All NED packages are files with the `.signed.bin` extension named using the following rule: `ncs---.signed.bin`. -2. Place the NED package in the `/tmp/ned-package-store` directory and configure the environment variable `NSO_RUNDIR` to point to the NSO runtime directory. -3. Unpack the NED package and verify its signature. The result of the unpacking is a `tar.gz` file with the same name as the `.bin` file. -4. Untar the `.tar.gz` file. The result is a subdirectory named like `-.`. -5. Install the NED on NSO, using the `ncs-setup` tool. -6. Finally, open an NSO CLI session and load the new NED package. -{% endtab %} - -{% tab title="NED Installation on System Install" %} -{% hint style="info" %} -This procedure below broadly outlines the steps needed to install a NED package on a [System Install](../installation-and-deployment/system-install.md). For most up-to-date and specific installation instructions, consult the `README.md` supplied with the NED. -{% endhint %} - -General instructions to install a NED package: - -1. Download the latest production-grade version of the NED from [software.cisco.com](https://software.cisco.com) using the URLs provided on your NED license certificates. All NED packages are files with the `.signed.bin` extension named using the following rule: `ncs---.signed.bin`. -2. Place the NED package in the `/tmp/ned-package-store` directory. -3. Unpack the NED package and verify its signature. The result of the unpacking is a `.tar.gz` file with the same name as the `.bin` file. -4. Perform an NSO backup before installing the new NED package. -5. Start an NSO CLI session. -6. Fetch the NED package. -7. Install the NED package (add the argument `replace-existing` if a previous version has been loaded). -8. Finally, load the NED package. -{% endtab %} -{% endtabs %} - -## Configuring a Device with an Installed NED - -Once a NED has been installed in NSO, the next step is to create and configure device entries that use this NED. The basic steps for configuring a device instance using a newly installed NED package are described in this section. Only the most basic configuration steps are covered here. Many NEDs also require additional custom configuration to be operational. This applies in particular to Generic NEDs. Information about configuration and such additional configuration can be found in the files `README.md` and `README-ned-settings.md` bundled with the NED package. - -The following info is necessary to proceed with the basic setup of a device instance in NSO: - -* NED ID of the new NED. -* Connection information for the device to connect to (address and port). -* Authentication information to the device (username and password). - -The general steps to configure a device with a NED are: - -1. Start an NSO CLI session. -2. Enter the configuration mode. -3. Configure a new authentication group to be used for this device. -4. Configure the new device instance, such as its IP address, port, etc. -5. Check the `README.md` and `README-ned-settings.md` bundled with the NED package for further information on additional settings to make the NED fully operational. -6. Commit the configuration. - -## Managing Cisco-provided Third Party YANG NEDs - -The third-party YANG NED type is a special category of the generic NED type targeted for devices supporting protocols like NETCONF, RESTCONF, and gNMI. As the name implies, this NED category is used for cases where the device YANG models are not implemented or maintained by the Cisco NSO NED Team. Instead, the YANG models are typically provided by the device vendor itself or by organizations like IETF, IEEE, ONF, or OpenConfig. - -A third-party YANG NED package is delivered from the software.cisco.com portal without any device YANG models included. It is required that the models are first downloaded, followed by a rebuild and reload of the package, before the NED can become fully operational. This task needs to be performed by the NED user. - -Detailed NED-specific instructions to manage Cisco-provided third-party YANG NEDs are provided in the respective READMEs. - -## NED Migration - -If you upgrade a managed device (such as installing a new firmware), the device data model can change in a significant way. If this is the case, you usually need to use a different and newer NED with an updated YANG model. - -When the changes in the NED are not backward compatible, the NED is assigned a new ned-id to avoid breaking existing code. On the plus side, this allows you to use both versions of the NED at the same time, so some devices can use the new version and some can use the old one. As a result, there is no need to upgrade all devices at the same time. The downside is, NSO doesn't know the two NEDs are related and will not perform any upgrade on its own due to different ned-ids. Instead, you must manually change the NED of a managed device through a NED migration. - -{% hint style="info" %} -For third-party NEDs, the end user is required to configure the NED ID and also be aware of the backward incompatibilities. -{% endhint %} - -Migration is required when upgrading a NED and the NED-ID changes, which is signified by a change in either the first or the second number in the NED package version. For example, if you're upgrading the existing `router-nc-1.0.1` NED to `router-nc-1.2.0` or `router-nc-2.0.2`, you must perform NED migration. On the other hand, upgrading to `router-nc-1.0.2` or `router-nc-1.0.3` retains the same ned-id and you can upgrade the `router-1.0.1` package in place, directly replacing it with the new one. However, note that some third-party, non-Cisco packages may not adhere to this standard versioning convention. In that case, you must check the ned-id values to see whether migration is needed. - -

Sample NED Package Versioning

- -A potential issue with a new NED is that it can break an existing service or other packages that rely on it. To help service developers and operators verify or upgrade the service code, NSO provides additional options of migration tooling for identifying the paths and service instances that may be impacted. Therefore, ensure that all the other packages are compatible with the new NED before you start migrating devices. - -To prepare for the NED migration process, first, load the new NED package into NSO with either `packages reload` or `packages add` command. Then, use the `show packages` command to verify that both NEDs, the new and the old, are present. Finally, you may perform the migration of devices either one by one or multiple at a time. - -Depending on your operational policies, this may be done during normal operations and does not strictly require a maintenance window, as the migration only reads from and doesn't write to a network device. Still, it is recommended that you create an NSO backup before proceeding. - -Note that changing a ned-id also affects device templates if you use them. To make existing device templates compatible with the new ned-id, you can use the `copy` action. It will copy the configuration used for one ned-id to another, as long as the schema nodes used haven't changed between the versions. The following example demonstrates the `copy` action usage: - -```bash -admin@ncs(config)# devices template acme-ntp ned-id router-nc-1.0 -copy ned-id router-nc-1.2 -``` - -For individual devices, use the `/devices/device/migrate` action, with the `new-ned-id` parameter. Without additional options, the command will read and update the device configuration in NSO. As part of this process, NSO migrates all the configuration and service meta-data. Use the `dry-run` option to see what the command would do and `verbose` to list all impacted service instances. - -You may also use the `no-networking` option to prevent NSO from generating any southbound traffic towards the device. In this case, only the device configuration in the CDB is used for the migration but then NSO can't know if the device is in sync. Afterward, you must use the **compare-config** or the **sync-from** action to remedy this. - -For migrating multiple devices, use the `/devices/migrate` action, which takes the same options. However, with this action, you must also specify the `old-ned-id`, which limits the migration to devices using the old NED. You can further restrict the action with the `device` parameter, selecting only specific devices. - -It is possible for a NED migration to fail if the new NED is not entirely backward compatible with the old one and the device has an active configuration that is incompatible with the new NED version. In such cases, NSO will produce an error with the YANG constraint that is not satisfied. Here, you must first manually adjust the device configuration to make it compatible with the new NED, and then you can perform the migration as usual. - -Depending on what changes are introduced by the migration and how these impact the services, it might be good to `re-deploy` the affected services before removing the old NED package. It is especially recommended in the following cases: - -* When the service touches a list key that has changed. As long as the old schema is loaded, NSO is able to perform an upgrade. -* When a namespace that was used by the service has been removed. The service diffset, that is, the recorded configuration changes created by the service, will no longer be valid. The diffset is needed for the correct `get-modifications` output, `deep-check-sync`, and similar operations. - -## Migrating from Legacy to Third-party NED - -{% hint style="info" %} -This section uses `juniper-junos_nc` as an example third-party NED. The process is generally same and applicable to other third-party NEDs. -{% endhint %} - -NSO has supported Junos devices from early on. The legacy Junos NED is NETCONF-based, but as Junos devices did not provide YANG modules in the past, complex NSO machinery translated Juniper's XML Schema Description (XSD) files into a single YANG module. This was an attempt to aggregate several Juniper device modules/versions. - -Juniper nowadays provides YANG modules for Junos devices. Junos YANG modules can be downloaded from the device and used directly in NSO with the new `juniper-junos_nc` NED. - -By downloading the YANG modules using `juniper-junos_nc` NED tools and rebuilding the NED, the NED can provide full coverage immediately when the device is updated instead of waiting for a new legacy NED release. - -This guide describes how to replace the legacy `juniper-junos` NED and migrate NSO applications to the `juniper-junos_nc` NED using the NSO MPLS VPN example from the NSO examples collection as a reference. - -Prepare the example: - -1. Add the `juniper-junos` and `juniper-junos_nc` NED packages to the example. -2. Configure the connection to the Junos device. -3. Add the MPLS VPN service configuration to the simulated network, including the Junos device using the legacy `juniper-junos` NED. - -Adapting the service to the `juniper-junos_nc` NED: - -1. Un-deploy MPLS VPN service instances with `no-networking`. -2. Delete Junos device config with `no-networking`. -3. Set the Junos device to NETCONF/YANG compliant mode. -4. Download the compliant YANG models, build, and reload the `juniper-junos_nc` NED package. -5. Switch the ned-id for the Junos device to the `juniper-junos_nc` NED package. -6. Sync from the Junos device to get the compliant Junos device config. -7. Update the MPLS VPN service to handle the difference between the non-compliant and compliant configurations belonging to the service. -8. Re-deploy the MPLS VPN service instances with `no-networking` to make the MPLS VPN service instances own the device configuration again. - -{% hint style="info" %} -If applying the steps for this example on a production system, you should first take a backup using the `ncs-backup` tool before proceeding. -{% endhint %} - -### Prepare the Example - -This guide uses the MPLS VPN example in Python from the NSO example set under [examples.ncs/service-management/mpls-vpn-python](https://github.com/NSO-developer/nso-examples/tree/6.6/service-management/mpls-vpn-python) to demonstrate porting an existing application to use the `juniper-junos_nc` NED. The simulated Junos device is replaced with a Junos vMX 21.1R1.11 container, but other NETCONF/YANG-compliant Junos versions also work. - -### **Add the `juniper-junos` and `juniper-junos_nc` NED Packages** - -The first step is to add the latest `juniper-junos` and `juniper-junos_nc` NED packages to the example's package directory. The NED tar-balls must be available and downloaded from your [https://software.cisco.com/download/home](https://software.cisco.com/download/home) account to the `mpls-vpn-python` example directory. Replace the `NSO_VERSION` and `NED_VERSION` variables with the versions you use: - -```bash -$ cd $NCS_DIR/examples.ncs/service-management/mpls-vpn-python -$ cp ./ncs-NSO_VERSION-juniper-junos-NED_VERSION.tar.gz packages/ -$ cd packages -$ tar xfz ../ncs-NSO_VERSION-juniper-junos_nc-NED_VERSION.tar.gz -$ cd - -``` - -Build and start the example: - -```bash -$ make all start -``` - -### **Configure the Connection to the Junos Device** - -Replace the netsim device connection configuration in NSO with the configuration for connecting to the Junos device. Adjust the `USER_NAME`, `PASSWORD`, and `HOST_NAME/IP_ADDR` variables and the timeouts as required for the Junos device you are using with this example: - -```bash -$ ncs_cli -u admin -C -admin@ncs# config -admin@ncs(config)# devices authgroups group juniper umap admin remote-name USER_NAME \ - remote-password PASSWORD -admin@ncs(config)# devices device pe2 authgroup juniper address HOST_NAME/IP_ADDR port 830 -admin@ncs(config)# devices device pe2 connect-timeout 240 -admin@ncs(config)# devices device pe2 read-timeout 240 -admin@ncs(config)# devices device pe2 write-timeout 240 -admin@ncs(config)# commit -admin@ncs(config)# end -admin@ncs# exit -``` - -Open a CLI terminal or use NETCONF on the Junos device to verify that the `rfc-compliant` and `yang-compliant` modes are not yet enabled. Examples: - -```bash -$ ssh USER_NAME@HOST_NAME/IP_ADDR -junos> configure -junos# show system services netconf -ssh; -``` - -Or: - -```bash -$ netconf-console -s plain -u USER_NAME -p PASSWORD --host=HOST_NAME/IP_ADDR \ - --port=830 --get-config - --subtree-filter=-<<<' - - - - - - ' - - - - - - - - - - - - - - - -``` - -The `rfc-compliant` and `yang-compliant` nodes must not be enabled yet for the legacy Junos NED to work. If enabled, delete in the Junos CLI or using NETCONF. A netconf-console example: - -```bash -$ netconf-console -s plain -u USER_NAME -p PASSWORD --host=HOST_NAME/IP_ADDR --port=830 - --db=candidate - --edit-config=- <<<' - - - - - - - - - ' - -$ netconf-console -s plain -u USER_NAME -p PASSWORD --host=HOST_NAME/IP_ADDR \ - --port=830 --commit -``` - -Back to the NSO CLI to upgrade the legacy `juniper-junos` NED to the latest version: - -```bash -$ ncs_cli -u admin -C -admin@ncs# config -admin@ncs(config)# devices device pe2 ssh fetch-host-keys -admin@ncs(config)# devices device pe2 migrate new-ned-id juniper-junos-nc-NED_VERSION -admin@ncs(config)# devices sync-from -admin@ncs(config)# end -``` - -### **Add the MPLS VPN Service Configuration to the Simulated Network** - -Turn off `autowizard` and `complete-on-space` to make it possible to paste configs: - -```cli -admin@ncs# autowizard false -admin@ncs# complete-on-space false -``` - -The example service config for two MPLS VPNs where the endpoints have been selected to pass through the `PE` node `PE2`, which is a Junos device: - -``` -vpn l3vpn ikea -as-number 65101 -endpoint branch-office1 - ce-device ce1 - ce-interface GigabitEthernet0/11 - ip-network 10.7.7.0/24 - bandwidth 6000000 -! -endpoint branch-office2 - ce-device ce4 - ce-interface GigabitEthernet0/18 - ip-network 10.8.8.0/24 - bandwidth 300000 -! -endpoint main-office - ce-device ce0 - ce-interface GigabitEthernet0/11 - ip-network 10.10.1.0/24 - bandwidth 12000000 -! -qos qos-policy GOLD -! -vpn l3vpn spotify -as-number 65202 -endpoint branch-office1 - ce-device ce5 - ce-interface GigabitEthernet0/1 - ip-network 10.2.3.0/24 - bandwidth 10000000 -! -endpoint branch-office2 - ce-device ce3 - ce-interface GigabitEthernet0/4 - ip-network 10.4.5.0/24 - bandwidth 20000000 -! -endpoint main-office - ce-device ce2 - ce-interface GigabitEthernet0/8 - ip-network 10.0.1.0/24 - bandwidth 40000000 -! -qos qos-policy GOLD -! -``` - -To verify that the traffic passes through `PE2`: - -```cli -admin@ncs(config)# commit dry-run outformat native -``` - -Toward the end of this lengthy output, observe that some config changes are going to the `PE2` device using the `http://xml.juniper.net/xnm/1.1/xnm` legacy namespace: - -``` -device { - name pe2 - data - - - - - test-then-set - rollback-on-error - - - - - - xe-0/0/2 - - 102 - Link to CE / ce5 - GigabitEthernet0/1 - - -
- 192.168.1.22/30 -
-
-
- 102 -
-
-
- ... -``` - -Looks good. Commit to the network: - -```cli -admin@ncs(config)# commit -``` - -### Adapting the Service to the `juniper-junos_nc` NED - -Now that the service's configuration is in place using the legacy `juniper-junos` NED to configure the `PE2` Junos device, proceed and switch to using the `juniper-junos_nc` NED with `PE2` instead. The service template and Python code will need a few adaptations. - -### **Un-deploy MPLS VPN Services Instances with `no-networking`** - -To keep the NSO service meta-data information intact when bringing up the service with the new `juniper-junos_nc` NED, first `un-deploy` the service instances in NSO, only keeping the configuration on the devices: - -```cli -admin@ncs(config)# vpn l3vpn * un-deploy no-networking -``` - -### **Delete Junos Device Config with `no-networking`** - -First, save the legacy Junos non-compliant mode device configuration to later diff against the compliant mode config: - -```cli -admin@ncs(config)# show full-configuration devices device pe2 config \ - configuration | display xml | save legacy.xml -``` - -Delete the `PE2` configuration in NSO to prepare for retrieving it from the device in a NETCONF/YANG compliant format using the new NED: - -```cli -admin@ncs(config)# no devices device pe2 config -admin@ncs(config)# commit no-networking -admin@ncs(config)# end -admin@ncs# exit -``` - -### **Set the Junos Device to NETCONF/YANG Compliant Mode** - -Using the Junos CLI: - -```bash -$ ssh USER_NAME@HOST_NAME/IP_ADDR -junos> configure -junos# set system services netconf rfc-compliant -junos# set system services netconf yang-compliant -junos# show system services netconf -ssh; -rfc-compliant; -ÿang-compliant; -junos# commit -``` - -Or, using the NSO `netconf-console` tool: - -```bash -$ netconf-console -s plain -u USER_NAME -p PASSWORD --host=HOST_NAME/IP_ADDR --port=830 \ - --db=candidate - --edit-config=- <<<' - - - - - - - - - ' - -$ netconf-console -s plain -u USER_NAME -p PASSWORD --host=HOST_NAME/IP_ADDR --port=830 \ - --commit -``` - -### **Switch the NED ID for the Junos Device to the `juniper-junos_nc` NED Package** - -```bash -$ ncs_cli -u admin -C -admin@ncs# config -admin@ncs(config)# devices device pe2 device-type generic ned-id juniper-junos_nc-gen-1.0 -admin@ncs(config)# commit -admin@ncs(config)# end -``` - -### **Download the Compliant YANG models, Build, and Load the `juniper-junos_nc` NED Package** - -The `juniper-junos_nc` NED is delivered without YANG modules, enabling populating it with device-specific YANG modules. The YANG modules are retrieved directly from the Junos device: - -```bash -$ ncs_cli -u admin -C -admin@ncs# devices device pe2 connect -admin@ncs# devices device pe2 rpc rpc-get-modules get-modules -admin@ncs# exit -``` - -See the `juniper-junos_nc` `README` for more options and details. - -Build the YANG modules retrieved from the Junos device with the `juniper-junos_nc` NED: - -```bash -$ make -C packages/juniper-junos_nc-gen-1.0/src -``` - -Reload the packages to load the `juniper-junos_nc` NED with the added YANG modules: - -```bash -$ ncs_cli -u admin -C -admin@ncs# packages reload -``` - -### **Sync From the Junos Device to Get the Device Configuration in NETCONF/YANG Compliant Format** - -```cli -admin@ncs# devices device pe2 sync-from -``` - -### **Update the MPLS VPN Service** - -The service must be updated to handle the difference between the Junos device's non-compliant and compliant configuration. The NSO service uses Python code to configure the Junos device using a service template. One way to find the required updates to the template and code is to check the difference between the non-compliant and compliant configurations for the parts covered by the template. - -

Side by Side, Running Config on the Left, Template on the Right.

- -Checking the `packages/l3vpn/templates/l3vpn-pe.xml` service template Junos device part under the legacy `http://xml.juniper.net/xnm/1.1/xnm` namespace, you can observe that it configures `interfaces`, `routing-instances`, `policy-options`, and `class-of-service`. - -You can save the NETCONF/YANG compliant Junos device configuration and diff it against the non-compliant configuration from the previously stored `legacy.xml` file: - -```cli -admin@ncs# show running-config devices device pe2 config configuration \ - | display xml | save new.xml -``` - -Examining the difference between the configuration in the `legacy.xml` and `new.xml` files for the parts covered by the service template: - -1. There is no longer a single namespace covering all configurations. The configuration is now divided into multiple YANG modules with a namespace for each. -2. The `/configuration/policy-options/policy-statement/then/community` node choice identity is no longer provided with a leaf named `key1`. Instead, the leaf name is `choice-ident`, and a `choice-value` leaf is set. -3. The `/configuration/class-of-service/interfaces/interface/unit/shaping-rate/rate` leaf format has changed from using an `int32` value to a string with either no suffix or a "k", "m" or "g" suffix. This differs from the other devices controlled by the template, so a new template `BW_SUFFIX` variable set from the Python code is needed. - -To enable the template to handle a Junos device in NETCONF/YANG compliant mode, add the following to the `packages/l3vpn/templates/l3vpn-pe.xml` service template: - -```xml - - -
-+ -+ -+ -+ -+ {$PE_INT_NAME} -+ -+ -+ -+ -+ {$VLAN_ID} -+ Link to CE / {$CE} - {$CE_INT_NAME} -+ {$VLAN_ID} -+ -+ -+
-+ {$LINK_PE_ADR}/{$LINK_PREFIX} -+
-+
-+
-+
-+
-+
-+ -+ -+ {/name} -+ vrf -+ -+ {$PE_INT_NAME}.{$VLAN_ID} -+ -+ -+ {/as-number}:1 -+ -+ {/name}-IMP -+ {/name}-EXP -+ -+ -+ -+ -+ -+ {/name} -+ {$LINK_PE_ADR} -+ {/as-number} -+ -+ 100 -+ -+ -+ {$LINK_CE_ADR} -+ -+ -+ -+ -+ -+ -+ -+ -+ {/name}-EXP -+ -+ bgp -+ -+ -+ -+ add -+ -+ {/name}-comm-exp -+ -+ -+ -+ -+ -+ {/name}-IMP -+ -+ bgp -+ {/name}-comm-imp -+ -+ -+ -+ -+ -+ -+ {/name}-comm-imp -+ target:{/as-number}:1 -+ -+ -+ {/name}-comm-exp -+ target:{/as-number}:1 -+ -+ -+ -+ -+ -+ {$PE_INT_NAME} -+ -+ {$VLAN_ID} -+ -+ {$BW_SUFFIX} -+ -+ -+ -+ -+ -+
-
-
- -``` - -The Python file changes to handle the new `BW_SUFFIX` variable to generate a string with a suffix instead of an `int32`: - -```bash -# of the service. These functions can be useful e.g. for -# allocations that should be stored and existing also when the -# service instance is removed. -+ -+ @staticmethod -+ def int32_to_numeric_suffix_str(val): -+ for suffix in ["", "k", "m", "g", ""]: -+ suffix_val = int(val / 1000) -+ if suffix_val * 1000 != val: -+ return str(val) + suffix -+ val = suffix_val -+ -@ncs.application.Service.create -def cb_create(self, tctx, root, service, proplist): - # The create() callback is invoked inside NCS FASTMAP and must -``` - -Code that uses the function and set the string to the service template: - -``` - tv.add('LOCAL_CE_NET', getIpAddress(endpoint.ip_network)) - tv.add('CE_MASK', getNetMask(endpoint.ip_network)) -+ tv.add('BW_SUFFIX', self.int32_to_numeric_suffix_str(endpoint.bandwidth)) - tv.add('BW', endpoint.bandwidth) - tmpl = ncs.template.Template(service) - tmpl.apply('l3vpn-pe', tv) -``` - -After making the changes to the service template and Python code, reload the updated package(s): - -```bash -$ ncs_cli -u admin -C -admin@ncs# packages reload -``` - -### **Re-deploy the MPLS VPN Service Instances** - -The service instances need to be re-deployed to own the device configuration again: - -```cli -admin@ncs# vpn l3vpn * re-deploy no-networking -``` - -The service is now in sync with the device configuration stored in NSO CDB: - -```cli -admin@ncs# vpn l3vpn * check-sync -vpn l3vpn ikea check-sync -in-sync true -vpn l3vpn spotify check-sync -in-sync true -``` - -When re-deploying the service instances, any issues with the added service template section for the compliant Junos device configuration, such as the added namespaces and nodes, are discovered. - -As there is no validation for the rate leaf string with a suffix in the Junos device model, no errors are discovered if it is provided in the wrong format until updating the Junos device. Comparing the device configuration in NSO with the configuration on the device shows such inconsistencies without having to test the configuration with the device: - -```cli -admin@ncs# devices device pe2 compare-config -``` - -If there are issues, correct them and redo the `re-deploy no-networking` for the service instances. - -When all issues have been resolved, the service configuration is in sync with the device configuration, and the NSO CDB device configuration matches to the configuration on the Junos device: - -```bash -$ ncs_cli -u admin -C -admin@ncs# vpn l3vpn * re-deploy -``` - -The NSO service instances are now in sync with the configuration on the Junos device using the `juniper-junos_nc` NED. - -## Revision Merge Functionality - -The YANG modeling language supports the notion of a module `revision`. It allows users to distinguish between different versions of a module, so the module can evolve over time. If you wish to use a new revision of a module for a managed device, for example, to access new features, you generally need to create a new NED. - -When a model evolves quickly and you have many devices that require the use of a lot of different revisions, you will need to maintain a high number of NEDs, which are mostly the same. This can become especially burdensome during NSO version upgrades, when all NEDs may need to be recompiled. - -When a YANG module is only updated in a backward-compatible way (following the upgrade rules in RFC6020 or RFC7950), the NSO compiler, `ncsc`, allows you to pack multiple module revisions into the same package. This way, a single NED with multiple device model revisions can be used, instead of multiple NEDs. Based on the capabilities exchange, NSO will then use the correct revision for communication with each device. - -However, there is a major downside to this approach. While the exact revision is known for each communication session with the managed device, the device model in NSO does not have that information. For that reason, the device model always uses the latest revision. When pushing configuration to a device that only supports an older revision, NSO silently drops the unsupported parts. This may have surprising results, as the NSO copy can contain configuration that is not really supported on the device. Use the `no-revision-drop` commit parameter when you want to make sure you are not committing config that is not supported by a device. - -If you still wish to use this functionality, you can create a NED package with the `ncs-make-package --netconf-ned` command as you would otherwise. However, the supplied source YANG directory should contain YANG modules with different revisions. The files should follow the _`module-or-submodule-name`_`@`_`revision-date`_`.yang` naming convention, as specified in the RFC6020. Some versions of the compiler require you to use the `--no-fail-on-warnings` option with the `ncs-make-package` command or the build process may fail. - -The [examples.ncs/device-management/ned-yang-revision](https://github.com/NSO-developer/nso-examples/tree/6.6/device-management/ned-yang-revision) example shows how you can perform a YANG model upgrade. The original, 1.0 version of the router NED uses the `router@2020-02-27.yang` YANG model. First, it is updated to the version 1.0.1 `router@2020-09-18.yang` using a revision merge approach. This is possible because the changes are backward-compatible. - -In the second part of the example, the updates in `router@2022-01-25.yang` introduce breaking changes, therefore the version is increased to 1.1 and a different NED-ID is assigned to the NED. In this case, you can't use revision merge and the usual NED migration procedure is required. diff --git a/administration/management/package-mgmt.md b/administration/management/package-mgmt.md deleted file mode 100644 index cf4fbbff..00000000 --- a/administration/management/package-mgmt.md +++ /dev/null @@ -1,265 +0,0 @@ ---- -description: Perform package management tasks. ---- - -# Package Management - -All user code that needs to run in NSO must be part of a package. A package is basically a directory of files with a fixed file structure or a tar archive with the same directory layout. A package consists of code, YANG modules, etc., that are needed to add an application or function to NSO. Packages are a controlled way to manage loading and versions of custom applications. - -Network Element Drivers (NEDs) are also packages. Each NED allows NSO to manage a network device of a specific type. Except for third-party YANG NED packages which do not contain a YANG device model by default (and must be downloaded and fixed before adding to the package), a NED typically contains a device YANG model and the code, specifying how NSO should connect to the device. For NETCONF devices, NSO includes built-in tools to help you build a NED, as described in [NED Administration](ned-administration.md), that you can use if needed. Otherwise, a third-party YANG NED, if available, should be used instead. Vendors, in some cases, provide the required YANG device models but not the entire NED. In practice, all NSO instances use at least one NED. The set of used NED packages depends on the number of different device types the NSO manages. - -When NSO starts, it searches for packages to load. The `ncs.conf` parameter `/ncs-config/load-path` defines a list of directories. At initial startup, NSO searches these directories for packages and copies the packages to a private directory tree in the directory defined by the `/ncs-config/state-dir` parameter in `ncs.conf`, and loads and starts all the packages found. On subsequent startups, NSO will by default only load and start the copied packages. The purpose of this procedure is to make it possible to reliably load new or updated packages while NSO is running, with a fallback to the previously existing version of the packages if the reload should fail. - -In a System Install of NSO, packages are always installed (normally through symbolic links) in the `packages` subdirectory of the run directory, i.e. by default `/var/opt/ncs/packages`, and the private directory tree is created in the `state` subdirectory, i.e. by default `/var/opt/ncs/state`. - -## Loading Packages - -Loading of new or updated packages (as well as removal of packages that should no longer be used) can be requested via the `reload` action - from the NSO CLI: - -```bash -admin@ncs# packages reload -reload-result { - package cisco-ios - result true -} -``` - -This request makes NSO copy all packages found in the load path to a temporary version of its private directory, and load the packages from this directory. If the loading is successful, this temporary directory will be made permanent, otherwise, the temporary directory is removed and NSO continues to use the previous version of the packages. Thus when updating packages, always update the version in the load path, and request that NSO does the reload via this action. - -If the package changes include modified, added, or deleted `.fxs` files or `.ccl` files, NSO needs to run a data model upgrade procedure, also called a CDB upgrade. NSO provides a `dry-run` option to `packages reload` action to test the upgrade without committing the changes. Using a reload dry-run, you can tell if a CDB upgrade is needed or not. - -The `report all-schema-changes` option of the reload action instructs NSO to produce a report of how the current data model schema is being changed. Combined with a dry run, the report allows you to verify the modifications introduced with the new versions of the packages before actually performing the upgrade. - -For a data model upgrade, including a dry run, all transactions must be closed. In particular, users having CLI sessions in configure mode must exit to operational mode. If there are ongoing commit queue items, and the `wait-commit-queue-empty` parameter is supplied, it will wait for the items to finish before proceeding with the reload. During this time, it will not allow the creation of any new transactions. Hence, if one of the queue items fails with `rollback-on-error` option set, the commit queue's rollback will also fail, and the queue item will be locked. In this case, the reload will be canceled. A manual investigation of the failure is needed in order to proceed with the reload. - -While the data model upgrade is in progress, all transactions are closed and new transactions are not allowed. This means that starting a new management session, such as a CLI or SSH connection to the NSO, will also fail, producing an error that the node is in upgrade mode. - -By default, the `reload` action will (when needed) wait up to 10 seconds for the commit queue to empty (if the `wait-commit-queue-empty` parameter is entered) and reload to start. - -If there are still open transactions at the end of this period, the upgrade will be canceled and the reload operation will fail. The `max-wait-time` and `timeout-action` parameters to the action can modify this behavior. For example, to wait for up to 30 seconds, and forcibly terminate any transactions that still remain open after this period, we can invoke the action as: - -```cli -admin@ncs# packages reload max-wait-time 30 timeout-action kill -``` - -Thus the default values for these parameters are `10` and `fail`, respectively. In case there are no changes to `.fxs` or .`ccl` files, the reload can be carried out without the data model upgrade procedure, and these parameters are ignored since there is no need to close open transactions. - -When reloading packages, NSO will give a warning when the upgrade looks suspicious, i.e., may break some functionality. Note that this is not a strict upgrade validation, but only intended as a hint to the NSO administrator early in the upgrade process that something might be wrong. Currently, the following scenarios will trigger the warnings: - -* One or more namespaces are removed by the upgrade. The consequence of this is all data belonging to this namespace is permanently deleted from CDB upon upgrade. This may be intended in some scenarios, in which case it is advised to proceed with overriding warnings as described below. -* There are source `.java` files found in the package, but no matching `.class` files in the jars loaded by NSO. This likely means that the package has not been compiled. -* There are matching `.class` files with modification time older than the source files, which hints that the source has been modified since the last time the package was compiled. This likely means that the package was not re-compiled the last time the source code was changed. - -If a warning has been triggered it is a strong recommendation to fix the root cause. If all of the warnings are intended, it is possible to proceed with `packages reload force` command. - -In some specific situations, upgrading a package with newly added custom validation points in the data model may produce an error similar to `no registration found for callpoint NEW-VALIDATION/validate` or simply `application communication failure`, resulting in an aborted upgrade. See [New Validation Points](../../development/core-concepts/using-cdb.md#cdb.upgrade-add-vp) on how to proceed. - -In some cases, we may want NSO to do the same operation as the `reload` action at NSO startup, i.e. copy all packages from the load path before loading, even though the private directory copy already exists. This can be achieved in the following ways: - -* Setting the shell environment variable `$NCS_RELOAD_PACKAGES` to `true`. This will make NSO do the copy from the load path on every startup, as long as the environment variable is set. In a System Install, NSO is typically started as a `systemd` system service, and `NCS_RELOAD_PACKAGES=true` can be set in `/etc/ncs/ncs.systemd.conf` temporarily to reload the packages. -* Giving the option `--with-package-reload` to the `ncs` command when starting NSO. This will make NSO do the copy from the load path on this particular startup, without affecting the behavior on subsequent startups. -* If warnings are encountered when reloading packages at startup using one of the options above, the recommended way forward is to fix the root cause as indicated by the warnings as mentioned before. If the intention is to proceed with the upgrade without fixing the underlying cause for the warnings, it is possible to force the upgrade using `NCS_RELOAD_PACKAGES`=`force` environment variable or `--with-package-reload-force` option. - -Always use one of these methods when upgrading to a new version of NSO in an existing directory structure, to make sure that new packages are loaded together with the other parts of the new system. - -## Redeploying Packages - -If it is known in advance that there were no data model changes, i.e. none of the `.fxs` or `.ccl` files changed, and none of the shared JARs changed in a Java package, and the declaration of the components in the `package-meta-data.xml` is unchanged, then it is possible to do a lightweight package upgrade, called package redeploy. Package redeploy only loads the specified package, unlike packages reload which loads all of the packages found in the load-path. - -```bash -admin@ncs# packages package mserv redeploy -result true -``` - -Redeploying a package allows you to reload updated or load new templates, reload private JARs for a Java package, or reload the Python code which is a part of this package. Only the changed part of the package will be reloaded, e.g. if there were no changes to Python code, but only templates, then the Python VM will not be restarted, but only templates reloaded. The upgrade is not seamless however as the old templates will be unloaded for a short while before the new ones are loaded, so any user of the template during this period of time will fail; the same applies to changed Java or Python code. It is hence the responsibility of the user to make sure that the services or other code provided by the package is unused while it is being redeployed. - -The `package redeploy` will return `true` if the package's resulting status after the redeploy is `up`. Consequently, if the result of the action is `false`, then it is advised to check the operational status of the package in the package list. - -```bash -admin@ncs# show packages package mserv oper-status -oper-status file-load-error -oper-status error-info "template3.xml:2 Unknown servicepoint: templ42-servicepoint" -``` - -## Adding NED Packages - -Unlike a full `packages reload` operation, new NED packages can be loaded into the system without disrupting existing transactions. This is only possible for new packages, since these packages don't yet have any instance data. - -The operation is performed through the `/packages/add` action. No additional input is necessary. The operation scans all the load-paths for any new NED packages and also verifies the existing packages are still present. If packages are modified or deleted, the operation will fail. - -Each NED package defines `ned-id`, an identifier that is used in selecting the NED for each managed device. A new NED package is therefore a package with a ned-id value that is not already in use. - -In addition, the system imposes some additional constraints, so it is not always possible to add just any arbitrary NED. In particular, NED packages can also contain one or more shared data models, such as NED settings or operational data for private use by the NED, that are not specific to each version of NED package but rather shared between all versions. These are typically placed outside any mount point (device-specific data model), extending the NSO schema directly. So, if a NED defines schema nodes outside any mount point, there must be no changes to these nodes if they already exist. - -Adding a NED package with a modified shared data model is therefore not allowed and all shared data models are verified to be identical before a NED package can be added. If they are not, the `/packages/add` action will fail and you will have to use the `/packages/reload` command. - -```bash -admin@ncs# packages add -add-result { - package router-nc-1.1 - result true -} -``` - -The command returns `true` if the package's resulting status after deployment is `up`. Likewise, if the result for a package is `false`, then the package was added but its code has not started successfully and you should check the operational status of the package with the `show packages package oper-status` command for additional information. You may then use the `/packages/package/redeploy` action to retry deploying the package's code, once you have corrected the error. - -{% hint style="info" %} -In a high-availability setup, you can perform this same operation on all the nodes in the cluster with a single `packages ha sync and-add` command. -{% endhint %} - -## Managing Packages - -In a System Install of NSO, management of pre-built packages is supported through a number of actions. This support is not available in a Local Install, since it is dependent on the directory structure created by the System Install. Please refer to the YANG submodule `$NCS_DIR/src/ncs/yang/tailf-ncs-software.yang` for the full details of the functionality described in this section. - -### Actions - -Actions are provided to list local packages, to fetch packages from the file system, and to install or deinstall packages: - -* `software packages list [...]`: List local packages, categorized into loaded, installed, and installable. The listing can be restricted to only one of the categories - otherwise, each package listed will include the category for the package. -* `software packages fetch package-from-file `: Fetch a package by copying it from the file system, making it installable. -* `software packages install package [...]`: Install a package, making it available for loading via the `packages reload` action, or via a system restart with package reload. The action ensures that only one version of the package is installed - if any version of the package is installed already, the `replace-existing` option can be used to deinstall it before proceeding with the installation. -* `software packages deinstall package `: Deinstall a package, i.e. remove it from the set of packages available for loading. - -There is also an `upload` action that can be used via NETCONF or REST to upload a package from the local host to the NSO host, making it installable there. It is not feasible to use in the CLI or Web UI, since the actual package file contents is a parameter for the action. It is also not suitable for very large (more than a few megabytes) packages, since the processing of action parameters is not designed to deal with very large values, and there is a significant memory overhead in the processing of such values. - -## More on Package Management - -NSO Packages contain data models and code for a specific function. It might be NED for a specific device, a service application like MPLS VPN, a WebUI customization package, etc. Packages can be added, removed, and upgraded in run-time. A common task is to add a package to NSO to support a new device type or upgrade an existing package when the device is upgraded. - -(We assume you have the example up and running from the previous section). Currently installed packages can be viewed with the following command: - -```bash -admin@ncs# show packages -packages package cisco-ios - package-version 3.0 - description "NED package for Cisco IOS" - ncs-min-version [ 3.0.2 ] - directory ./state/packages-in-use/1/cisco-ios-cli-3.0 - component upgrade-ned-id - upgrade java-class-name com.tailf.packages.ned.ios.UpgradeNedId - component cisco-ios - ned cli ned-id cisco-ios-cli-3.0 - ned cli java-class-name com.tailf.packages.ned.ios.IOSNedCli - ned device vendor Cisco -NAME VALUE ---------------------- -show-tag interface - - oper-status up -``` - -So the above command shows that NSO currently has one package, the NED for Cisco IOS. - -NSO reads global configuration parameters from `ncs.conf`. More on NSO configuration later in this guide. By default, it tells NSO to look for packages in a `packages` directory where NSO was started. Using the [examples.ncs/device-management/simulated-cisco-ios](https://github.com/NSO-developer/nso-examples/tree/6.6/device-management/simulated-cisco-ios) example to demonstrate: - -```bash -$ pwd -examples.ncs/device-management/simulated-cisco-ios -$ NONINTERACTIVE=1 ./demo.sh -$ ls packages/ -cisco-ios-cli-3.0 -$ ls packages/cisco-ios-cli-3.0 -doc -load-dir -netsim -package-meta-data.xml -private-jar -shared-jar -src -``` - -As seen above a package is a defined file structure with data models, code, and documentation. NSO comes with a few ready-made example packages: `$NCS_DIR/packages/`. Also, there is a library of packages available from Tail-f, especially for supporting specific devices. - -### Adding and Upgrading a Package - -Assume you would like to add support for Nexus devices to the example. Nexus devices have different data models and another CLI flavor. There is a package for that in `$NCS_DIR/packages/neds/nexus`. - -We can keep NSO running all the time, but we will stop the network simulator to add the Nexus devices to the simulator. - -```bash -$ ncs-netsim stop -``` - -Add the nexus package to the NSO runtime directory by creating a symbolic link: - -```bash -$ cd $NCS_DIR/examples.ncs/device-management/simulated-cisco-ios/packages -$ ln -s $NCS_DIR/packages/neds/cisco-nx-cli-3.0 cisco-nx-cli-3.0 -$ ls -l -... -cisco-nx-cli-3.0 -> $NCS_DIR/packages/neds/cisco-nx-cli-3.0 -``` - -The package is now in place, but until we tell NSO to look for package changes nothing happens: - -```bash - admin@ncs# show packages packages package - cisco-ios ... admin@ncs# packages reload - ->>> System upgrade is starting. ->>> Sessions in configure mode must exit to operational mode. ->>> No configuration changes can be performed until upgrade has -completed. ->>> System upgrade has completed successfully. -reload-result { - package cisco-ios - result true -} -reload-result { - package cisco-nx - result true -} -``` - -So after the `packages reload` operation NSO also knows about Nexus devices. The reload operation also takes any changes to existing packages into account. The data store is automatically upgraded to cater to any changes like added attributes to existing configuration data. - -### Simulating the New Device - -```bash -$ ncs-netsim add-to-network cisco-nx-cli-3.0 2 n -$ ncs-netsim list -ncs-netsim list for examples.ncs/device-management/simulated-cisco-ios/netsim - -name=c0 ... -name=c1 ... -name=c2 ... -name=n0 ... -name=n1 ... - - -$ ncs-netsim start -DEVICE c0 OK STARTED -DEVICE c1 OK STARTED -DEVICE c2 OK STARTED -DEVICE n0 OK STARTED -DEVICE n1 OK STARTED -$ ncs-netsim cli-c n0 -n0#show running-config -no feature ssh -no feature telnet -fex 101 - pinning max-links 1 -! -fex 102 - pinning max-links 1 -! -nexus:vlan 1 -! -... -``` - -### Adding the New Devices to NSO - -We can now add these Nexus devices to NSO according to the below sequence: - -```bash -admin@ncs(config)# devices device n0 device-type cli ned-id cisco-nx-cli-3.0 -admin@ncs(config-device-n0)# port 10025 -admin@ncs(config-device-n0)# address 127.0.0.1 -admin@ncs(config-device-n0)# authgroup default -admin@ncs(config-device-n0)# state admin-state unlocked -admin@ncs(config-device-n0)# commit -admin@ncs(config-device-n0)# top -admin@ncs(config)# devices device n0 sync-from -result true -``` diff --git a/administration/management/system-management/README.md b/administration/management/system-management/README.md deleted file mode 100644 index 20a2e6fa..00000000 --- a/administration/management/system-management/README.md +++ /dev/null @@ -1,763 +0,0 @@ ---- -description: Perform NSO system management and configuration. ---- - -# System Management - -NSO consists of a number of modules and executable components. These executable components will be referred to by their command-line name, e.g. `ncs`, `ncs-netsim`, `ncs_cli`, etc. `ncs` is used to refer to the executable, the running daemon. - -## Starting NSO - -When NSO is started, it reads its configuration file and starts all subsystems configured to start (such as NETCONF, CLI, etc.). - -By default, NSO starts in the background without an associated terminal. It is recommended to use a [System Install](../../installation-and-deployment/system-install.md) when installing NSO for production deployment. This will create an `init` script that starts NSO when the system boots, and makes NSO start the service manager. - -## Licensing NSO - -NSO is licensed using Cisco Smart Licensing. To register your NSO instance, you need to enter a token from your Cisco Smart Software Manager account. For more information on this topic, see [Cisco Smart Licensing](cisco-smart-licensing.md)_._ - -## Configuring NSO - -NSO is configured in the following two ways: - -* Through its configuration file, `ncs.conf`. -* Through whatever data is configured at run-time over any northbound, for example, turning on trace using the CLI. - -### `ncs.conf` File - -The configuration file `ncs.conf` is read at startup and can be reloaded. Below is an example of the most common settings. It is included here as an example and should be self-explanatory. See [ncs.conf](../../../resources/man/ncs.conf.5.md) in Manual Pages for more information. Important configuration settings are: - -* `load-path`: where NSO should look for compiled YANG files, such as data models for NEDs or Services. -* `db-dir`: the directory on disk that CDB uses for its storage and any temporary files being used. It is also the directory where CDB searches for initialization files. This should be a local disk and not NFS mounted for performance reasons. -* Various log settings. -* AAA configuration. -* Rollback file directory and history length. -* Enabling north-bound interfaces like REST, and WebUI. -* Enabling of High-Availability mode. - -The `ncs.conf` file is described in the [NSO Manual Pages](../../../resources/man/ncs.conf.5.md). There is a large number of configuration items in `ncs.conf`, most of them have sane default values. The `ncs.conf` file is an XML file that must adhere to the `tailf-ncs-config.yang` model. If we start the NSO daemon directly, we must provide the path to the NCS configuration file as in: - -```bash -# ncs -c /etc/ncs/ncs.conf -``` - -However, in a System Install, `systemd` is typically used to start NSO, and it will pass the appropriate options to the `ncs` command. Thus, NSO is started with the command: - -```bash -# systemctl nso start -``` - -It is possible to edit the `ncs.conf` file, and then tell NSO to reload the edited file without restarting the daemon as in: - -```bash -# ncs --reload -``` - -This command also tells NSO to close and reopen all log files, which makes it suitable to use from a system like `logrotate`. - -In this section, some of the important configuration settings will be described and discussed. - -### Exposed Interfaces - -NSO allows access through a number of different interfaces, depending on the use case. In the default configuration, clients can access the system locally through an unauthenticated IPC socket (with the `ncs*` family of commands, port 4569) and plain (non-HTTPS) HTTP web server (port 8080). Additionally, the system enables remote access through SSH-secured NETCONF and CLI (ports 2022 and 2024). - -We strongly encourage you to review and customize the exposed interfaces to your needs in the `ncs.conf` configuration file. In particular, set: - -* `/ncs-config/webui/match-host-name` to `true`. -* `/ncs-config/webui/server-name` to the hostname of the server. -* `/ncs-config/webui/server-alias` to additional domains or IP addresses used for serving HTTP(S). - -If you decide to allow remote access to the web server, make sure you use TLS-secured HTTPS instead of HTTP and keep `match-host-name` enabled. Not doing so exposes you to security risks. - -{% hint style="info" %} -Using `/ncs-config/webui/match-host-name = true` requires you to use the configured hostname when accessing the server. Web browsers do this automatically but you may need to set the `Host` header when performing requests programmatically using an IP address instead of the hostname. -{% endhint %} - -To additionally secure IPC access, refer to [Restricting Access to the IPC Socket](../../advanced-topics/ipc-connection.md#restricting-access-to-the-ipc-socket). - -For more details on individual interfaces and their use, see [Northbound APIs](../../../development/core-concepts/northbound-apis/). - -### Dynamic Configuration - -Let's look at all the settings that can be manipulated through the NSO northbound interfaces. NSO itself has a number of built-in YANG modules. These YANG modules describe the structure that is stored in CDB. Whenever we change anything under, say `/devices/device`, it will change the CDB, but it will also change the configuration of NSO. We call this dynamic configuration since it can be changed at will through all northbound APIs. - -We summarize the most relevant parts below: - -```cli -ncs@ncs(config)# -Possible completions: - aaa AAA management, users and groups - cluster Cluster configuration - devices Device communication settings - java-vm Control of the NCS Java VM - nacm Access control - packages Installed packages - python-vm Control of the NCS Python VM - services Global settings for services, (the services themselves might be augmented somewhere else) - session Global default CLI session parameters - snmp Top-level container for SNMP related configuration and status objects. - snmp-notification-receiver Configure reception of SNMP notifications - software Software management - ssh Global SSH connection configuration -``` - -#### **`tailf-ncs.yang` Module** - -This is the most important YANG module that is used to control and configure NSO. The module can be found at: `$NCS_DIR/src/ncs/yang/tailf-ncs.yang` in the release. Everything in that module is available through the northbound APIs. The YANG module has descriptions for everything that can be configured. - -`tailf-common-monitoring2.yang` and `tailf-ncs-monitoring2.yang` are two modules that are relevant to monitoring NSO. - -### Built-in or External SSH Server - -NSO has a built-in SSH server which makes it possible to SSH directly into the NSO daemon. Both the NSO northbound NETCONF agent and the CLI need SSH. To configure the built-in SSH server we need a directory with server SSH keys - it is specified via `/ncs-config/aaa/ssh-server-key-dir` in `ncs.conf`. We also need to enable `/ncs-config/netconf-north-bound/transport/ssh` and `/ncs-config/cli/ssh` in `ncs.conf`. In a System Install, `ncs.conf` is installed in the "config directory", by default `/etc/ncs`, with the SSH server keys in `/etc/ncs/ssh`. - -### Run-time Configuration - -There are also configuration parameters that are more related to how NSO behaves when talking to the devices. These reside in `devices global-settings`. - -```cli -admin@ncs(config)# devices global-settings -Possible completions: - backlog-auto-run Auto-run the backlog at successful connection - backlog-enabled Backlog requests to non-responding devices - commit-queue - commit-retries Retry commits on transient errors - connect-timeout Timeout in seconds for new connections - ned-settings Control which device capabilities NCS uses - out-of-sync-commit-behaviour Specifies the behaviour of a commit operation involving a device that is out of sync with NCS. - read-timeout Timeout in seconds used when reading data - report-multiple-errors By default, when the NCS device manager commits data southbound and when there are errors, we only - report the first error to the operator, this flag makes NCS report all errors reported by managed - devices - trace Trace the southbound communication to devices - trace-dir The directory where trace files are stored - write-timeout Timeout in seconds used when writing - data -``` - -## User Management - -Users are configured at the path `aaa authentication users`. - -```cli -admin@ncs(config)# show full-configuration aaa authentication users user -aaa authentication users user admin - uid 1000 - gid 1000 - password $1$GNwimSPV$E82za8AaDxukAi8Ya8eSR. - ssh_keydir /var/ncs/homes/admin/.ssh - homedir /var/ncs/homes/admin -! -aaa authentication users user oper - uid 1000 - gid 1000 - password $1$yOstEhXy$nYKOQgslCPyv9metoQALA. - ssh_keydir /var/ncs/homes/oper/.ssh - homedir /var/ncs/homes/oper -!... -``` - -Access control, including group memberships, is managed using the NACM model (RFC 6536). - -```cli -admin@ncs(config)# show full-configuration nacm -nacm write-default permit -nacm groups group admin - user-name [ admin private ] -! -nacm groups group oper - user-name [ oper public ] -! -nacm rule-list admin - group [ admin ] - rule any-access - action permit - ! -! -nacm rule-list any-group - group [ * ] - rule tailf-aaa-authentication - module-name tailf-aaa - path /aaa/authentication/users/user[name='$USER'] - access-operations read,update - action permit - ! -``` - -### Adding a User - -Adding a user includes the following steps: - -1. Create the user: `admin@ncs(config)# aaa authentication users user `. -2. Add the user to a NACM group: `admin@ncs(config)# nacm groups admin user-name `. -3. Verify/change access rules. - -It is likely that the new user also needs access to work with device configuration. The mapping from NSO users and corresponding device authentication is configured in `authgroups`. So, the user needs to be added there as well. - -```cli -admin@ncs(config)# show full-configuration devices authgroups -devices authgroups group default - umap admin - remote-name admin - remote-password $4$wIo7Yd068FRwhYYI0d4IDw== - ! - umap oper - remote-name oper - remote-password $4$zp4zerM68FRwhYYI0d4IDw== - ! -! -``` - -If the last step is forgotten, you will see the following error: - -```cli -jim@ncs(config)# devices device c0 config ios:snmp-server community fee -jim@ncs(config-config)# commit -Aborted: Resource authgroup for jim doesn't exist -``` - -## Monitoring NSO - -This section describes how to monitor NSO. See also [NSO Alarms](./#nso-alarms). - -Use the command `ncs --status` to get runtime information on NSO. - -### NSO Status - -Checking the overall status of NSO can be done using the shell: - -```bash -$ ncs --status -``` - -Or, in the CLI: - -```cli -ncs# show ncs-state -``` - -For details on the output see `$NCS_DIR/src/yang/tailf-common-monitoring2.yang`. - -Below is an overview of the output: - -
daemon-statusYou can see the NSO daemon mode, starting, phase0, phase1, started, stopping. The phase0 and phase1 modes are schema upgrade modes and will appear if you have upgraded any data models.
versionThe NSO version.
smpNumber of threads used by the daemon.
haThe High-Availability mode of the NCS daemon will show up here: secondary, primary, relay-secondary.
internal/callpoints

The next section is callpoints. Make sure that any validation points, etc. are registered. (The ncs-rfs-service-hook is an obsolete callpoint, ignore this one).

  • UNKNOWN code tries to register a call-point that does not exist in a data model.
  • NOT-REGISTERED a loaded data model has a call-point but no code has registered.

Of special interest is of course the servicepoints. All your deployed service models should have a corresponding service-point. For example:

servicepoints:
-  id=l3vpn-servicepoint daemonId=10 daemonName=ncs-dp-6-l3vpn:L3VPN
-  id=nsr-servicepoint daemonId=11 daemonName=ncs-dp-7-nsd:NSRService
-  id=vm-esc-servicepoint daemonId=12 daemonName=ncs-dp-8-vm-manager-esc:ServiceforVMstarting
-  id=vnf-catalogue-esc daemonId=13 daemonName=ncs-dp-9-vnf-catalogue-esc:ESCVNFCatalogueService
-
internal/cdbThe cdb section is important. Look for any locks. This might be a sign that a developer has taken a CDB lock without releasing it. The subscriber section is also important. A design pattern is to register subscribers to wait for something to change in NSO and then trigger an action. Reactive FASTMAP is designed around that. Validate that all expected subscribers are OK.
loaded-data-modelsThe next section shows all namespaces and YANG modules that are loaded. If you, for example, are missing a service model, make sure it is loaded.
cli, netconf, rest, snmp, webuiAll northbound agents like CLI, REST, NETCONF, SNMP, etc. are listed with their IP and port. So if you want to connect over REST, for example, you can see the port number here.
patchesLists any installed patches.
upgrade-modeIf the node is in upgrade mode, it is not possible to get any information from the system over NETCONF. Existing CLI sessions can get system information.
- -It is also important to look at the packages that are loaded. This can be done in the CLI with: - -``` -admin> show packages -packages package cisco-asa - package-version 3.4.0 - description "NED package for Cisco ASA" - ncs-min-version [ 3.2.2 3.3 3.4 4.0 ] - directory ./state/packages-in-use/1/cisco-asa - component upgrade-ned-id - upgrade java-class-name com.tailf.packages.ned.asa.UpgradeNedId - component ASADp - callback java-class-name [ com.tailf.packages.ned.asa.ASADp ] - component cisco-asa - ned cli ned-id cisco-asa - ned cli java-class-name com.tailf.packages.ned.asa.ASANedCli - ned device vendor Cisco -``` - -### Monitoring the NSO Daemon - -NSO runs the following processes: - -* **The daemon**: `ncs.smp`: this is the NCS process running in the Erlang VM. -* **Java VM**: `com.tailf.ncs.NcsJVMLauncher`: service applications implemented in Java run in this VM. There are several options on how to start the Java VM, it can be monitored and started/restarted by NSO or by an external monitor. See the [ncs.conf(5)](../../../resources/man/ncs.conf.5.md) Manual Page and the `java-vm` settings in the CLI. -* **Python VMs**: NSO packages can be implemented in Python. The individual packages can be configured to run a VM each or share a Python VM. Use the `show python-vm status current` to see current threads and `show python-vm status start` to see which threads were started at startup time. - -### Logging - -NSO has extensive logging functionality. Log settings are typically very different for a production system compared to a development system. Furthermore, the logging of the NSO daemon and the NSO Java VM/Python VM is controlled by different mechanisms. During development, we typically want to turn on the `developer-log`. The sample `ncs.conf` that comes with the NSO release has log settings suitable for development, while the `ncs.conf` created by a System Install are suitable for production deployment. - -NSO logs in `/logs` in your running directory, (depends on your settings in `ncs.conf`). You might want the log files to be stored somewhere else. See man `ncs.conf` for details on how to configure the various logs. Below is a list of the most useful log files: - -* `ncs.log` : NCS daemon log. See [Log Messages and Formats](log-messages-and-formats.md). Can be configured to Syslog. -* `ncserr.log.1`_,_ `ncserr.log.idx`_,_ `ncserr.log.siz`: if the NSO daemon has a problem. this contains debug information relevant to support. The content can be displayed with `ncs --printlog ncserr.log`. -* `audit.log`: central audit log covering all northbound interfaces. See [Log Messages and Formats](log-messages-and-formats.md). Can be configured to Syslog. -* `localhost:8080.access`: all HTTP requests to the daemon. This is an access log for the embedded Web server. This file adheres to the Common Log Format, as defined by Apache and others. This log is not enabled by default and is not rotated, i.e. use logrotate(8). Can be configured to Syslog. -* `devel.log`: developer-log is a debug log for troubleshooting user-written code. This log is enabled by default and is not rotated, i.e. use logrotate(8). This log shall be used in combination with the `java-vm` or `python-vm` logs. The user code logs in the VM logs and the corresponding library logs in `devel.log`. Disable this log in production systems. Can be configured to Syslog.\ - \ - You can manage this log and set its logging level in `ncs.conf`. - - ```xml - - true - - ${NCS_LOG_DIR}/devel.log - false - - - true - - - trace - ``` -* `ncs-java-vm`_._`log`_,_ `ncs-python-vm.log`: logger for code running in Java or Python VM, for example, service applications. Developers writing Java and Python code use this log (in combination with devel.log) for debugging. Both Java and Python log levels can be set from their respective VM settings in, for example, the CLI. - - ```cli - admin@ncs(config)# python-vm logging level level-info - admin@ncs(config)# java-vm java-logging logger com.tailf.maapi level level-info - ``` -* `netconf.log`_,_ `snmp.log`: Log for northbound agents. Can be configured to Syslog. -* `rollbackNNNNN`: All NSO commits generate a corresponding rollback file. The maximum number of rollback files and file numbering can be configured in `ncs.conf`. -* `xpath.trace`: XPATH is used in many places, for example, XML templates. This log file shows the evaluation of all XPATH expressions and can be enabled in the `ncs.conf`. - - ```xml - - true - ${NCS_LOG_DIR}/xpath.trace - - ``` - - To debug XPATH for a template, use the pipe target `debug` in the CLI instead. - - ```cli - admin@ncs(config)# commit | debug template - ``` -* `ned-cisco-ios-xr-pe1.trace` (for example): if device trace is turned on a trace file will be created per device. The file location is not configured in `ncs.conf` but is configured when the device trace is turned on, for example in the CLI. - - ```cli - admin@ncs(config)# devices device r0 trace pretty - ``` -* Progress trace log: When a transaction or action is applied, NSO emits specific progress events. These events can be displayed and recorded in a number of different ways, either in CLI with the pipe target `details` on a commit, or by writing it to a log file. You can read more about it in the [Progress Trace](../../../development/advanced-development/progress-trace.md). -* Transaction error log: log for collecting information on failed transactions that lead to either a CDB boot error or a runtime transaction failure. The default is `false` (disabled). More information about the log is available in the Manual Pages under [Configuration Parameters](../../../resources/man/ncs.conf.5.md#configuration-parameters) (see `logs/transaction-error-log`). -* Upgrade log: log containing information about CDB upgrade. The log is enabled by default and not rotated (i.e., use logrotate). With the NSO example set, the following examples populate the log in the `logs/upgrade.log` file: [examples.ncs/device-management/ned-yang-revision](https://github.com/NSO-developer/nso-examples/tree/6.6/device-management/ned-yang-revision), [examples.ncs/high-availability/upgrade-basic](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/upgrade-basic), [examples.ncs/high-availability/upgrade-cluster](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/upgrade-cluster), and [examples.ncs/service-management/upgrade-service](https://github.com/NSO-developer/nso-examples/tree/6.6/service-management/upgrade-service). More information about the log is available in the Manual Pages under [Configuration Parameters](../../../resources/man/ncs.conf.5.md#configuration-parameters) (see `logs/upgrade-log)`. - -### Syslog - -NSO can syslog to a local Syslog. See `man ncs.conf` how to configure the Syslog settings. All Syslog messages are documented in Log Messages. The `ncs.conf` also lets you decide which of the logs should go into Syslog: `ncs.log, devel.log, netconf.log, snmp.log, audit.log, WebUI access log`. There is also a possibility to integrate with `rsyslog` to log the NCS, developer, audit, netconf, SNMP, and WebUI access logs to syslog with the facility set to daemon in `ncs.conf`. For reference, see the `upgrade-l2` example [examples.ncs/high-availability/hcc](https://github.com/NSO-developer/nso-examples/tree/6.6/high-availability/hcc) . - -Below is an example of Syslog configuration: - -```xml - - daemon - - - - true - - ./logs/ncs.log - true - - - true - - -``` - -Log messages are described on the link below: - -{% content-ref url="log-messages-and-formats.md" %} -[log-messages-and-formats.md](log-messages-and-formats.md) -{% endcontent-ref %} - -### NSO Alarms - -NSO generates alarms for serious problems that must be remedied. Alarms are available over all the northbound interfaces and exist at the path `/alarms`. NSO alarms are managed as any other alarms by the general NSO Alarm Manager, see the specific section on the alarm manager in order to understand the general alarm mechanisms. - -The NSO alarm manager also presents a northbound SNMP view, alarms can be retrieved as an alarm table, and alarm state changes are reported as SNMP Notifications. See the "NSO Northbound" documentation on how to configure the SNMP Agent. - -This is also documented in the example [examples.ncs/northbound-interfaces/snmp-alarm](https://github.com/NSO-developer/nso-examples/tree/6.6/northbound-interfaces/snmp-alarm). - -Alarms are described on the link below: - -{% content-ref url="alarms.md" %} -[alarms.md](alarms.md) -{% endcontent-ref %} - -### Tracing in NSO - -Tracing enables observability across NSO operations by tagging requests with unique identifiers. NSO allows for using Trace Context (recommended) and Trace ID while the `label` commit parameter can be used to correlate events. These allow tracking of requests across service invocations, internal operations, and downstream device configurations. - -#### **Trace Context (Recommended)** - -NSO supports Trace Context based on the [W3C Trace Context specification](https://www.w3.org/TR/trace-context/), which is the recommended approach for distributed request tracing. This allows tracing information to flow between systems using standardized headers. - -When using Trace Context: - -* Trace information is carried in the `traceparent` and `tracestate` attributes. -* The trace ID is a UUID (RFC 4122) and is automatically generated and enforced. -* Trace Context is propagated automatically across NSO operations, including LSA setups and commit queues. -* There is no need to pass the trace ID manually as a commit parameter. -* It is supported across all major northbound protocols: NETCONF, RESTCONF, JSON-RPC, CLI, and MAAPI. -* Trace data appears in logs and trace files, enabling consistent request tracking across services and systems. - -{% hint style="info" %} -When Trace Context is used, NSO handles tracing internally in compliance with W3C standards. Using an explicit `trace-id` commit parameter is therefore neither needed nor recommended. -{% endhint %} - -#### Trace ID - -NSO can issue a unique Trace ID per northbound request, visible in logs and trace headers. This Trace ID can be used to follow the request from service invocation to configuration changes pushed to any device affected by the change. The Trace ID may either be passed in from an external client or generated by NSO. Note that: - -* Trace ID is enabled by default. -* Trace ID is propagated downwards in [LSA](../../advanced-topics/layered-service-architecture.md) setups and is fully integrated with commit queues. -* Trace ID can be passed to NSO over NETCONF, RESTCONF, JSON-RPC, CLI, or MAAPI as a commit parameter. -* If Trace ID is not given as a commit parameter, NSO will generate one. - -The generated Trace ID is an array of 16 random bytes, encoded as a 32-character hexadecimal string, in accordance with [Trace ID](https://www.w3.org/TR/trace-context/#trace-id). NSO also accepts arbitrary strings, but the UUID format (as per [RFC 4122](https://datatracker.ietf.org/doc/html/rfc4122), a 128-bit value formatted as a 36-character hyphenated string: xxxxxxxx-xxxx-Mxxx-Nxxx-xxxxxxxxxxxx, e.g., `550e8400-e29b-41d4-a716-446655440000`) is the preferred approach for creating Trace IDs. - -For RESTCONF requests, this generated Trace ID will be communicated back to the requesting client as an HTTP header called `X-Cisco-NSO-Trace-ID`. The `trace-id` query parameter can also be used with RPCs and actions to relay a trace-id from northbound requests. - -For NETCONF, the Trace ID will be returned as an attribute called `trace-id`. - -Trace ID will appear in relevant log entries and trace file headers on the form `trace-id=...`. - -## Disaster Management - -This section describes a number of disaster scenarios and recommends various actions to take in the different disaster variants. - -### NSO Fails to Start - -CDB keeps its data in four files `A.cdb`, `C.cdb`, `O.cdb` and `S.cdb`. If NSO is stopped, these four files can be copied, and the copy is then a full backup of CDB. - -Furthermore, if neither files exist in the configured CDB directory, CDB will attempt to initialize from all files in the CDB directory with the suffix `.xml`. - -Thus, there exist two different ways to re-initiate CDB from a previously known good state, either from `.xml` files or from a CDB backup. The `.xml` files would typically be used to reinstall factory defaults whereas a CDB backup could be used in more complex scenarios. - -If the `S.cdb` file has become inconsistent or has been removed, all commit queue items will be removed, and devices not yet processed out of sync. For such an event, appropriate alarms will be raised on the devices and any service instance that has unprocessed device changes will be set in the failed state. - -When NSO starts and fails to initialize, the following exit codes can occur: - -* Exit codes 1 and 19 mean that an internal error has occurred. A text message should be in the logs, or if the error occurred at startup before logging had been activated, on standard error (standard output if NSO was started with `--foreground --verbose`). Generally, the message will only be meaningful to the NSO developers, and an internal error should always be reported to support. -* Exit codes 2 and 3 are only used for the NCS control commands (see the section COMMUNICATING WITH NCS in the [ncs(1)](../../../resources/man/ncs.1.md) in Manual Pages manual page) and mean that the command failed due to timeout. Code 2 is used when the initial connect to NSO didn't succeed within 5 seconds (or the `TryTime` if given), while code 3 means that the NSO daemon did not complete the command within the time given by the `--timeout` option. -* Exit code 10 means that one of the init files in the CDB directory was faulty in some way — further information in the log. -* Exit code 11 means that the CDB configuration was changed in an unsupported way. This will only happen when an existing database is detected, which was created with another configuration than the current in `ncs.conf`. -* Exit code 13 means that the schema change caused an upgrade, but for some reason, the upgrade failed. Details are in the log. The way to recover from this situation is either to correct the problem or to re-install the old schema (`fxs`) files. -* Exit code 14 means that the schema change caused an upgrade, but for some reason the upgrade failed, corrupting the database in the process. This is rare and usually caused by a bug. To recover, either start from an empty database with the new schema, or re-install the old schema files and apply a backup. -* Exit code 15 means that `A.cdb` or `C.cdb` is corrupt in a non-recoverable way. Remove the files and re-start using a backup or init files. -* Exit code 16 means that CDB ran into an unrecoverable file error (such as running out of space on the device while performing journal compaction). -* Exit code 20 means that NSO failed to bind a socket. -* Exit code 21 means that some NSO configuration file is faulty. More information is in the logs. -* Exit code 22 indicates an NSO installation-related problem, e.g., that the user does not have read access to some library files, or that some file is missing. - -If the NSO daemon starts normally, the exit code is 0. - -If the AAA database is broken, NSO will start but with no authorization rules loaded. This means that all write access to the configuration is denied. The NSO CLI can be started with a flag `ncs_cli --noaaa` that will allow full unauthorized access to the configuration. - -### NSO Failure After Startup - -NSO attempts to handle all runtime problems without terminating, e.g., by restarting specific components. However, there are some cases where this is not possible, described below. When NSO is started the default way, i.e. as a daemon, the exit codes will of course not be available, but see the `--foreground` option in the [ncs(1)](../../../resources/man/ncs.1.md) Manual Page. - -* **Out of memory**: If NSO is unable to allocate memory, it will exit by calling abort(3). This will generate an exit code, as for reception of the SIGABRT signal - e.g. if NSO is started from a shell script, it will see 134, as the exit code (128 + the signal number). -* **Out of file descriptors for accept(2)**: If NSO fails to accept a TCP connection due to lack of file descriptors, it will log this and then exit with code 25. To avoid this problem, make sure that the process and system-wide file descriptor limits are set high enough, and if needed configure session limits in `ncs.conf`. The out-of-file descriptors issue may also manifest itself in that applications are no longer able to open new file descriptors.\ - \ - In many Linux systems, the default limit is 1024, but if we, for example, assume that there are four northbound interface ports, CLI, RESTCONF, SNMP, WebUI/JSON-RPC, or similar, plus a few hundred IPC ports, x 1024 == 5120. But one might as well use the next power of two, 8192, to be on the safe side. - - \ - Several application issues can contribute to consuming extra ports. In the scope of an NSO application that could, for example, be a script application that invokes CLI command or a callback daemon application that does not close the connection socket as it should. - - A commonly used command for changing the maximum number of open file descriptors is `ulimit -n [limit]`. Commands such as `netstat` and `lsof` can be useful to debug file descriptor-related issues. - -### Transaction Commit Failure - -When the system is updated, NSO executes a two-phase commit protocol towards the different participating databases including CDB. If a participant fails in the `commit()` phase although the participant succeeded in the preparation phase, the configuration is possibly in an inconsistent state. - -When NSO considers the configuration to be in an inconsistent state, operations will continue. It is still possible to use NETCONF, the CLI, and all other northbound management agents. The CLI has a different prompt which reflects that the system is considered to be in an inconsistent state and also the Web UI shows this: - -``` - -- WARNING ------------------------------------------------------ - Running db may be inconsistent. Enter private configuration mode and - install a rollback configuration or load a saved configuration. - ------------------------------------------------------------------ -``` - -The MAAPI API has two interface functions that can be used to set and retrieve the consistency status, those are `maapi_set_running_db_status()` and `maapi_get_running_db_status()` corresponding. This API can thus be used to manually reset the consistency state. The only alternative to reset the state to a consistent state is by reloading the entire configuration. - -## Backup and Restore - -All parts of the NSO installation can be backed up and restored with standard file system backup procedures. - -The most convenient way to do backup and restore is to use the `ncs-backup` command. In that case, the following procedure is used. - -### Take a Backup - -NSO Backup backs up the database (CDB) files, state files, config files, and rollback files from the installation directory. To take a complete backup (for disaster recovery), use: - -```bash -# ncs-backup -``` - -The backup will be stored in the "run directory", by default `/var/opt/ncs`, as `/var/opt/ncs/backups/ncs-VERSION@DATETIME.backup`. - -For more information on backup, refer to the [ncs-backup(1)](../../../resources/man/ncs-backup.1.md) in Manual Pages. - -### Restore a Backup - -NSO Restore is performed if you would like to switch back to a previous good state or restore a backup. - -It is always advisable to stop NSO before performing a restore. - -1. First stop NSO if NSO is not stopped yet. - - ``` - systemctl stop ncs - ``` -2. Restore the backup. - - ```bash - ncs-backup --restore - ``` - - \ - Select the backup to be restored from the available list of backups. The configuration and database with run-time state files are restored in `/etc/ncs` and `/var/opt/ncs`. -3. Start NSO. - - ``` - systemctl start ncs - ``` - -## Rollbacks - -NSO supports creating rollback files during the commit of a transaction that allows for rolling back the introduced changes. Rollbacks do not come without a cost and should be disabled if the functionality is not going to be used. Enabling rollbacks impacts both the time it takes to commit a change and requires sufficient storage on disk. - -Rollback files contain a set of headers and the data required to restore the changes that were made when the rollback was created. One of the header fields includes a unique rollback ID that can be used to address the rollback file independent of the rollback numbering format. - -The use of rollbacks from the supported APIs and the CLI is documented in the documentation for the given API. - -### `ncs.conf` Config for Rollback - -As described [earlier](./#configuring-nso), NSO is configured through the configuration file, `ncs.conf`. In that file, we have the following items related to rollbacks: - -* `/ncs-config/rollback/enabled`: If set to `true`, then a rollback file will be created whenever the running configuration is modified. -* `/ncs-config/rollback/directory`: Location where rollback files will be created. -* `/ncs-config/rollback/history-size`: The number of old rollback files to save. - -## Troubleshooting - -New users can face problems when they start to use NSO. If you face an issue, reach out to our support team regardless if your problem is listed here or not. - -{% hint style="success" %} -A useful tool in this regard is the `ncs-collect-tech-report` tool, which is the Bash script that comes with the product. It collects all log files, CDB backup, and several debug dumps as a TAR file. Note that it works only with a System Install. - -```bash -root@linux:/# ncs-collect-tech-report --full -``` -{% endhint %} - -Some noteworthy issues are covered here. - -
- -Installation Problems: Error Messages During Installation - -* **Error** - - ``` - tar: Skipping to next header - gzip: stdin: invalid compressed data--format violated - ``` - -- **Impact**\ - The resulting installation is incomplete. - -* **Cause**\ - This happens if the installation program has been damaged, most likely because it has been downloaded in ASCII mode. - -- **Resolution**\ - Remove the installation directory. Download a new copy of NSO from our servers. Make sure you use binary transfer mode every step of the way. - -
- -
- -Problem Starting NSO: NSO Terminating with GLIBC Error - -* **Error** - - ``` - Internal error: Open failed: /lib/tls/libc.so.6: version - `GLIBC_2.3.4' not found (required by - .../lib/ncs/priv/util/syst_drv.so) - ``` - -- **Impact**\ - NSO terminates immediately with a message similar to the one above. - -* **Cause**\ - This happens if you are running on a very old Linux version. The GNU libc (GLIBC) version is older than 2.3.4, which was released in 2004. - -- **Resolution**\ - Use a newer Linux system, or upgrade the GLIBC installation. - -
- -
- -Problem in Running Examples: The netconf-console Program Fails - -* **Error**\ - You must install the Python SSH implementation Paramiko in order to use SSH. - -- **Impact**\ - Sending NETCONF commands and queries with `netconf-console` fails, while it works using `netconf-console-tcp`. - -* **Cause**\ - The `netconf-console` command is implemented using the Python programming language. It depends on the Python SSHv2 implementation Paramiko. Since you are seeing this message, your operating system doesn't have the Python module Paramiko installed. - -- **Resolution**\ - Install Paramiko using the instructions from [https://www.paramiko.org](https://www.paramiko.org/).\ - \ - When properly installed, you will be able to import the Paramiko module without error messages. - - ```bash - $ python - ... - >>> import paramiko - >>> - ``` - - \ - Exit the Python interpreter with Ctrl+D. - -* **Workaround**\ - A workaround is to use `netconf-console-tcp`. It uses TCP instead of SSH and doesn't require Paramiko. Note that TCP traffic is not encrypted. - -
- -
- -Problems Using and Developing Services - -If you encounter issues while loading service packages, creating service instances, or developing service models, templates, and code, you can consult the Troubleshooting section in [Implementing Services](../../../development/core-concepts/implementing-services.md). - -
- -### General Troubleshooting Strategies - -If you have trouble starting or running NSO, examples, or the clients you write, here are some troubleshooting tips. - -
- -Transcript - -When contacting support, it often helps the support engineer to understand what you are trying to achieve if you copy-paste the commands, responses, and shell scripts that you used to trigger the problem, together with any CLI outputs and logs produced by NSO. - -
- -
- -Source ENV Variables - -If you have problems executing `ncs` commands, make sure you source the `ncsrc` script in your NSO directory (your path may be different than the one in the example if you are using a local install), which sets the required environmental variables. - -```bash -$ source /etc/profile.d/ncs.sh -``` - -
- -
- -Log Files - -To find out what NSO is/was doing, browsing NSO log files is often helpful. In the examples, they are called `devel.log`, `ncs.log`, `audit.log`. If you are working with your own system, make sure that the log files are enabled in `ncs.conf`. They are already enabled in all the examples. You can read more about how to enable and inspect various logs in the [Logging](./#ug.ncs_sys_mgmt.logging) section. - -
- -
- -Verify HW Resources - -Both high CPU utilization and a lack of memory can negatively affect the performance of NSO. You can use commands such as `top` to examine resource utilization, and `free -mh` to see the amount of free and consumed memory. A common symptom of a lack of memory is NSO or Java-VM restarting. A sufficient amount of disk space is also required for CDB persistence and logs, so you can also check disk space with `df -h` command. In case there is enough space on the disk and you still encounter ENOSPC errors, check the inode usage with `df -i` command. - -
- -
- -Status - -NSO will give you a comprehensive status of daemon status, YANG modules, loaded packages, MIBs, active user sessions, CDB locks, and more if you run: - -```bash -$ ncs --status -``` - -NSO status information is also available as operational data under `/ncs-state`. - -
- -
- -Check Data Provider - -If you are implementing a data provider (for operational or configuration data), you can verify that it works for all possible data items using: - -```bash -$ ncs --check-callbacks -``` - -
- -
- -Debug Dump - -If you suspect you have experienced a bug in NSO, or NSO told you so, you can give Support a debug dump to help us diagnose the problem. It contains a lot of status information (including a full `ncs --status report`) and some internal state information. This information is only readable and comprehensible to the NSO development team, so send the dump to your support contact. A debug dump is created using: - -```bash -$ ncs --debug-dump mydump1 -``` - -Just as in CSI on TV, the information must be collected as soon as possible after the event. Many interesting traces will wash away with time, or stay undetected if there are lots of irrelevant facts in the dump. - -If NSO gets stuck while terminating, it can optionally create a debug dump after being stuck for 60 seconds. To enable this mechanism, set the environment variable `$NCS_DEBUG_DUMP_NAME` to a filename of your choice. - -
- -
- -Error Log - -Another thing you can do in case you suspect that you have experienced a bug in NSO is to collect the error log. The logged information is only readable and comprehensible to the NSO development team, so send the log to your support contact. The log actually consists of a number of files called `ncserr.log.*` - make sure to provide them all. - -
- -
- -System Dump - -If NSO aborts due to failure to allocate memory (see [Disaster Management](./#ug.ncs_sys_mgmt.disaster)), and you believe that this is due to a memory leak in NSO, creating one or more debug dumps as described above (before NSO aborts) will produce the most useful information for Support. If this is not possible, NSO will produce a system dump by default before aborting, unless `DISABLE_NCS_DUMP` is set. - -The default system dump file name is `ncs_crash.dump` and it could be changed by setting the environment variable `$NCS_DUMP` before starting NSO. The dumped information is only comprehensible to the NSO development team, so send the dump to your support contact. - -
- -
- -System Call Trace - -To catch certain types of problems, especially relating to system start and configuration, the operating system's system call trace can be invaluable. This tool is called `strace`/`ktrace`/`truss`. Please send the result to your support contact for a diagnosis. - -By running the instructions below. - -Linux: - -```bash -# strace -f -o mylog1.strace -s 1024 ncs ... -``` - -BSD: - -```bash -# ktrace -ad -f mylog1.ktrace ncs ... -# kdump -f mylog1.ktrace > mylog1.kdump -``` - -Solaris: - -```bash -# truss -f -o mylog1.truss ncs ... -``` - -
diff --git a/administration/management/system-management/alarms.md b/administration/management/system-management/alarms.md deleted file mode 100644 index 82fcb77f..00000000 --- a/administration/management/system-management/alarms.md +++ /dev/null @@ -1,693 +0,0 @@ -# Alarm Types - -``` -alarm-type - cdb-offload-threshold-too-low - certificate-expiration - ha-alarm - ha-node-down-alarm - ha-primary-down - ha-secondary-down - ncs-cluster-alarm - cluster-subscriber-failure - ncs-dev-manager-alarm - abort-error - auto-configure-failed - commit-through-queue-blocked - commit-through-queue-failed - commit-through-queue-failed-transiently - commit-through-queue-rollback-failed - configuration-error - connection-failure - final-commit-error - missing-transaction-id - ned-live-tree-connection-failure - out-of-sync - revision-error - ncs-package-alarm - package-load-failure - package-operation-failure - ncs-service-manager-alarm - service-activation-failure - ncs-snmp-notification-receiver-alarm - receiver-configuration-error - time-violation-alarm - transaction-lock-time-violation -``` - -## Alarm Type Descriptions - -
- -abort-error - -abort-error - -* **Initial Perceived Severity** - major -* **Description** - An error happened while aborting or reverting a transaction. Device's -configuration is likely to be inconsistent with the NCS CDB. -* **Recommended Action** - Inspect the configuration difference with compare-config, - resolve conflicts with sync-from or sync-to if any. -* **Clear Condition(s)** - If NCS achieves sync with the device, or receives a transaction - id for a netconf session towards the device, the alarm is cleared. -* **Alarm Message(s)** - * `Device {dev} is locked` - * `Device {dev} is southbound locked` - * `abort error` - -
- -
- -alarm-type - -alarm-type - -* **Description** - Base identity for alarm types. A unique identification of the -fault, not including the managed object. Alarm types are used -to identify if alarms indicate the same problem or not, for -lookup into external alarm documentation, etc. Different -managed object types and instances can share alarm types. If -the same managed object reports the same alarm type, it is to -be considered to be the same alarm. The alarm type is a -simplification of the different X.733 and 3GPP alarm IRP alarm -correlation mechanisms and it allows for hierarchical -extensions. -A 'specific-problem' can be used in addition to the alarm type -in order to have different alarm types based on information not -known at design-time, such as values in textual SNMP -Notification varbinds. - -
- -
- -auto-configure-failed - -auto-configure-failed - -* **Initial Perceived Severity** - warning -* **Description** - Device auto-configure exhausted its retry attempts trying -to connect and sync the device. -* **Recommended Action** - Make sure that NCS can connect to the device and then sync - the configuration. -* **Clear Condition(s)** - If NCS achieves sync with the device, the alarm is cleared. -* **Alarm Message(s)** - * `Auto-configure has exhausted its retry attempts` - -
- -
- -cdb-offload-threshold-too-low - -cdb-offload-threshold-too-low - -* **Initial Perceived Severity** - warning -* **Description** - The CDB offload threshold configuration is set too low, causing -the CDB memory footprint to reach the threshold even when there -is no offloadable data present in the memory. -* **Recommended Action** - If system memory is sufficient, increase the threshold value, otherwise - increase the system memory capacity. -* **Clear Condition(s)** - This alarm is cleared when CDB offload can lower the CDB memory - footprint below the configured threshold value. -* **Alarm Message(s)** - * `CDB offload threshold is too low` - -
- -
- -certificate-expiration - -certificate-expiration - -* **Description** - The certificate is nearing its expiry or has already expired. -The severity depends on the time left to expiry, it ranges from -warning to critical. -* **Recommended Action** - Replace certificate. -* **Clear Condition(s)** - This alarm is cleared when the certificate is no longer loaded. -* **Alarm Message(s)** - * `Certificate expires in less than {days} day(s)` - * `Certificate has expired` - -
- -
- -cluster-subscriber-failure - -cluster-subscriber-failure - -* **Initial Perceived Severity** - critical -* **Description** - Failure to establish a notification subscription towards -a remote node. -* **Recommended Action** - Verify IP connectivity between cluster nodes. -* **Clear Condition(s)** - This alarm is cleared if NCS succeeds to establish a - subscription towards the remote node, or when the subscription - is explicitly stopped. -* **Alarm Message(s)** - * `Failed to establish netconf notification - subscription to node ~s, stream ~s` - * `Commit queue items with remote nodes will not receive required - event notifications.` - -
- -
- -commit-through-queue-blocked - -commit-through-queue-blocked - -* **Initial Perceived Severity** - warning -* **Description** - A commit was queued behind a queue item waiting to be able to -connect to one of its devices. This is potentially dangerous -since one unreachable device can potentially fill up the commit -queue indefinitely. -* **Clear Condition(s)** - An alarm raised due to a transient error will be cleared - when NCS is able to reconnect to the device. -* **Alarm Message(s)** - * `Commit queue item ~p is blocked because item ~p cannot connect to ~s` - -
- -
- -commit-through-queue-failed - -commit-through-queue-failed - -* **Initial Perceived Severity** - critical -* **Description** - A queued commit failed. -* **Recommended Action** - Resolve with rollback if possible. -* **Clear Condition(s)** - This alarm is not cleared. -* **Alarm Message(s)** - * `Failed to authenticate towards device {device}: {reason}` - * `Device {dev} is locked` - * `{Reason}` - * `Device {dev} is southbound locked` - * `Commit queue item {CqId} rollback invoked` - * `Commit queue item {CqId} has failed: Operation failed because: - inconsistent database` - * `Remote commit queue item ~p cannot be unlocked: - cluster node not configured correctly` - -
- -
- -commit-through-queue-failed-transiently - -commit-through-queue-failed-transiently - -* **Initial Perceived Severity** - critical -* **Description** - A queued commit failed as it exhausted its retry attempts -on transient errors. -* **Recommended Action** - Resolve with rollback if possible. -* **Clear Condition(s)** - This alarm is not cleared. -* **Alarm Message(s)** - * `Failed to connect to device {dev}: {reason}` - * `Connection to {dev} timed out` - * `Failed to authenticate towards device {device}: {reason}` - * `The configuration database is locked for device {dev}: {reason}` - * `the configuration database is locked by session {id} {identification}` - * `the configuration database is locked by session {id} {identification}` - * `{Dev}: Device is locked in a {Op} operation by session {session-id}` - * `resource denied` - * `Commit queue item {CqId} rollback invoked` - * `Commit queue item {CqId} has failed: Operation failed because: - inconsistent database` - * `Remote commit queue item ~p cannot be unlocked: - cluster node not configured correctly` - -
- -
- -commit-through-queue-rollback-failed - -commit-through-queue-rollback-failed - -* **Initial Perceived Severity** - critical -* **Description** - Rollback of a commit-queue item failed. -* **Recommended Action** - Investigate the status of the device and resolve the - situation by issuing the appropriate action, i.e., service - redeploy or a sync operation. -* **Clear Condition(s)** - This alarm is not cleared. -* **Alarm Message(s)** - * `{Reason}` - -
- -
- -configuration-error - -configuration-error - -* **Initial Perceived Severity** - critical -* **Description** - Invalid configuration of NCS managed device, NCS cannot recognize -parameters needed to connect to device. -* **Recommended Action** - Verify that the configuration parameters defined in - tailf-ncs-devices.yang submodule are consistent for this device. -* **Clear Condition(s)** - The alarm is cleared when NCS reads the configuration - parameters for the device, and is raised again if the - parameters are invalid. -* **Alarm Message(s)** - * `Failed to resolve IP address for {dev}` - * `the configuration database is locked by session {id} {identification}` - * `{Reason}` - * `Resource {resource} doesn't exist` - -
- -
- -connection-failure - -connection-failure - -* **Initial Perceived Severity** - major -* **Description** - NCS failed to connect to a managed device before the timeout expired. -* **Recommended Action** - Verify address, port, authentication, check that the device is up - and running. If the error occurs intermittently, increase - connect-timeout. -* **Clear Condition(s)** - If NCS successfully reconnects to the device, the alarm is cleared. -* **Alarm Message(s)** - * `The connection to {dev} was closed` - * `Failed to connect to device {dev}: {reason}` - -
- -
- -final-commit-error - -final-commit-error - -* **Initial Perceived Severity** - critical -* **Description** - A managed device validated a configuration change, but failed to -commit. When this happens, NCS and the device are out of sync. -* **Recommended Action** - Reconcile by comparing and sync-from or sync-to. -* **Clear Condition(s)** - If NCS achieves sync with the device, the alarm is cleared. -* **Alarm Message(s)** - * `The connection to {dev} was closed` - * `External error in the NED implementation for device {dev}: {reason}` - * `Internal error in the NED NCS framework affecting device {dev}: {reason}` - -
- -
- -ha-alarm - -ha-alarm - -* **Description** - Base type for all alarms related to high availablity. -This is never reported, sub-identities for the specific -high availability alarms are used in the alarms. - -
- -
- -ha-node-down-alarm - -ha-node-down-alarm - -* **Description** - Base type for all alarms related to nodes going down in -high availablity. This is never reported, sub-identities -for the specific node down alarms are used in the alarms. - -
- -
- -ha-primary-down - -ha-primary-down - -* **Initial Perceived Severity** - critical -* **Description** - The node lost the connection to the primary node. -* **Recommended Action** - Make sure the HA cluster is operational, investigate why - the primary went down and bring it up again. -* **Clear Condition(s)** - This alarm is never automatically cleared and has to be cleared - manually when the HA cluster has been restored. -* **Alarm Message(s)** - * `Lost connection to primary due to: Primary closed connection` - * `Lost connection to primary due to: Tick timeout` - * `Lost connection to primary due to: code {Code}` - -
- -
- -ha-secondary-down - -ha-secondary-down - -* **Initial Perceived Severity** - critical -* **Description** - The node lost the connection to a secondary node. -* **Recommended Action** - Investigate why the secondary node went down, fix the - connectivity issue and reconnect the secondary to the - HA cluster. -* **Clear Condition(s)** - This alarm is cleared when the secondary node is reconnected - to the HA cluster. -* **Alarm Message(s)** - * `Lost connection to secondary` - -
- -
- -missing-transaction-id - -missing-transaction-id - -* **Initial Perceived Severity** - warning -* **Description** - A device announced in its NETCONF hello message that -it supports the transaction-id as defined in -http://tail-f.com/yang/netconf-monitoring. However when -NCS tries to read the transaction-id no data is returned. -The NCS check-sync feature will not work. This is usually -a case of misconfigured NACM rules on the managed device. -* **Recommended Action** - Verify NACM rules on the concerned device. -* **Clear Condition(s)** - If NCS successfully reads a transaction id for which - it had previously failed to do so, the alarm is cleared. -* **Alarm Message(s)** - * `{Reason}` - -
- -
- -ncs-cluster-alarm - -ncs-cluster-alarm - -* **Description** - Base type for all alarms related to cluster. -This is never reported, sub-identities for the specific -cluster alarms are used in the alarms. - -
- -
- -ncs-dev-manager-alarm - -ncs-dev-manager-alarm - -* **Description** - Base type for all alarms related to the device manager -This is never reported, sub-identities for the specific -device alarms are used in the alarms. - -
- -
- -ncs-package-alarm - -ncs-package-alarm - -* **Description** - Base type for all alarms related to packages. -This is never reported, sub-identities for the specific -package alarms are used in the alarms. - -
- -
- -ncs-service-manager-alarm - -ncs-service-manager-alarm - -* **Description** - Base type for all alarms related to the service manager -This is never reported, sub-identities for the specific -service alarms are used in the alarms. - -
- -
- -ncs-snmp-notification-receiver-alarm - -ncs-snmp-notification-receiver-alarm - -* **Description** - Base type for SNMP notification receiver Alarms. This is never -reported, sub-identities for specific SNMP notification receiver -alarms are used in the alarms. - -
- -
- -ned-live-tree-connection-failure - -ned-live-tree-connection-failure - -* **Initial Perceived Severity** - major -* **Description** - NCS failed to connect to a managed device using one of the optional -live-status-protocol NEDs. -* **Recommended Action** - Verify the configuration of the optional NEDs. - If the error occurs intermittently, increase connect-timeout. -* **Clear Condition(s)** - If NCS successfully reconnects to the managed device, - the alarm is cleared. -* **Alarm Message(s)** - * `The connection to {dev} was closed` - * `Failed to connect to device {dev}: {reason}` - -
- -
- -out-of-sync - -out-of-sync - -* **Initial Perceived Severity** - major -* **Description** - A managed device is out of sync with NCS. Usually it means that the -device has been configured out of band from NCS point of view. -* **Recommended Action** - Inspect the difference with compare-config, reconcile by - invoking sync-from or sync-to. -* **Clear Condition(s)** - If NCS achieves sync with the device, the alarm is cleared. -* **Alarm Message(s)** - * `Device {dev} is out of sync` - * `Out of sync due to no-networking or failed commit-queue commits.` - * `got: ~s expected: ~s.` - -
- -
- -package-load-failure - -package-load-failure - -* **Initial Perceived Severity** - critical -* **Description** - NCS failed to load a package. -* **Recommended Action** - Check the package for the reason. -* **Clear Condition(s)** - If NCS successfully loads a package for which an alarm - was previously raised, it will be cleared. -* **Alarm Message(s)** - * `failed to open file {file}: {str}` - * `Specific to the concerned package.` - -
- -
- -package-operation-failure - -package-operation-failure - -* **Initial Perceived Severity** - critical -* **Description** - A package has some problem with its operation. -* **Recommended Action** - Check the package for the reason. -* **Clear Condition(s)** - This alarm is not cleared. - -
- -
- -receiver-configuration-error - -receiver-configuration-error - -* **Initial Perceived Severity** - major -* **Description** - The snmp-notification-receiver could not setup its configuration, -either at startup or when reconfigured. SNMP notifications will now -be missed. -* **Recommended Action** - Check the error-message and change the configuration. -* **Clear Condition(s)** - This alarm will be cleared when the NCS is configured - to successfully receive SNMP notifications -* **Alarm Message(s)** - * `Configuration has errors.` - -
- -
- -revision-error - -revision-error - -* **Initial Perceived Severity** - major -* **Description** - A managed device arrived with a known module, but too new revision. -* **Recommended Action** - Upgrade the Device NED using the new YANG revision in order - to use the new features in the device. -* **Clear Condition(s)** - If all device yang modules are supported by NCS, - the alarm is cleared. -* **Alarm Message(s)** - * `The device has YANG module revisions not supported by - NCS. Use the /devices/device/check-yang-modules - action to check which modules that are not compatible.` - -
- -
- -service-activation-failure - -service-activation-failure - -* **Initial Perceived Severity** - critical -* **Description** - A service failed during re-deploy. -* **Recommended Action** - Corrective action and another re-deploy is needed. -* **Clear Condition(s)** - If the service is successfully redeployed, the alarm is cleared. -* **Alarm Message(s)** - * `Multiple device errors: -{str}` - -
- -
- -time-violation-alarm - -time-violation-alarm - -* **Description** - Base type for all alarms related to time violations. -This is never reported, sub-identities for the specific -time violation alarms are used in the alarms. - -
- -
- -transaction-lock-time-violation - -transaction-lock-time-violation - -* **Initial Perceived Severity** - warning -* **Description** - The transaction lock time exceeded its threshold and might be stuck -in the critical section. This threshold is configured in -/ncs-config/transaction-lock-time-violation-alarm/timeout. -* **Recommended Action** - Investigate if the transaction is stuck and possibly - interrupt it by closing the user session which it is - attached to. -* **Clear Condition(s)** - This alarm is cleared when the transaction has finished. -* **Alarm Message(s)** - * `Transaction lock time exceeded threshold.` - -
- diff --git a/administration/management/system-management/cisco-smart-licensing.md b/administration/management/system-management/cisco-smart-licensing.md deleted file mode 100644 index 780327df..00000000 --- a/administration/management/system-management/cisco-smart-licensing.md +++ /dev/null @@ -1,106 +0,0 @@ ---- -description: Manage purchase and licensing of Cisco software. ---- - -# Cisco Smart Licensing - -[Cisco Smart Licensing](https://www.cisco.com/web/ordering/smart-software-licensing/index.html) is a cloud-based approach to licensing, and it simplifies the purchase, deployment, and management of Cisco software assets. Entitlements are purchased through a Cisco account via Cisco Commerce Workspace (CCW) and are immediately deposited into a Smart Account for usage. This eliminates the need to install license files on every device. Products that are smart-enabled communicate directly to Cisco to report consumption. - -Cisco Smart Software Manager (CSSM) enables the management of software licenses and Smart Account from a single portal. The interface allows you to activate your product, manage entitlements, and renew and upgrade software. - -A functioning Smart Account is required to complete the registration process. For detailed information about CSSM, see [Cisco Smart Software Manager](https://www.cisco.com/c/en/us/buy/smart-accounts/software-manager.html). - -## Smart Accounts and Virtual Accounts - -A virtual account exists as a sub-account within the Smart Account. Virtual accounts are a customer-defined structure based on organizational layout, business function, geography, or any defined hierarchy. They are created and maintained by the Smart Account administrator(s). - -Visit [Cisco Cisco Software Central](https://software.cisco.com/) to learn about how to create and manage Smart Accounts. - -### Request a Smart Account - -The creation of a new Smart Account is a one-time event, and subsequent management of users is a capability provided through the tool. To request a Smart Account, visit [Cisco Cisco Software Central](https://software.cisco.com/) and take the following steps: - -1. After logging in, select **Request a Smart Account** in the Administration section. - -
-2. Select the type of Smart Account to create. There are two options: (a) Individual Smart Account requiring agreement to represent your company. By creating this Smart Account, you agree to authorization to create and manage product and service entitlements, users, and roles on behalf of your organization. (b) Create the account on behalf of someone else. - -
-3. Provide the required domain identifier and the preferred account name. - -
-4. The account request will be pending approval of the Account Domain Identifier. A subsequent email will be sent to the requester to complete the setup process. - -
- -### Adding Users to a Smart Account - -Smart Account user management is available in the **Administration** section of [Cisco Cisco Software Central](https://software.cisco.com/). Take the following steps to add a new user to a Smart Account: - -1. After logging in Select **Manage Smart Account** in the **Administration** section. - -
-2. Choose the **Users** tab. - -
-3. Select **New User** and follow the instructions in the wizard to add a new user. - -
- -### Create a License Registration Token - -1. To create a new token, log into CSSM and select the appropriate Virtual Account. - -
-2. Click on the **Smart Licenses** link to enter CSSM. - -
-3. In CSSM click on **New Token**. - -
-4. Follow the dialog to provide a description, expiration, and export compliance applicability before accepting the terms and responsibilities. Click on **Create Token** to continue. - -
-5. Click on the new token. - -
-6. Copy the token from the dialogue window into your clipboard. - -
-7. Go to the NSO CLI and provide the token to the `license smart register idtoken` command: - - ```cli - admin@ncs# license smart register idtoken YzY2YjFlOTYtOWYzZi00MDg1... - Registration process in progress. - Use the 'show license status' command to check the progress and result. - ``` - -### Notes on Configuring Smart Licensing - -* If `ncs.conf` contains configuration for any of java-executable, java-options, override-url/url, or proxy/url under the configure path `/ncs-config/smart-license/smart-agent/` any corresponding configuration done via the CLI is ignored. -* The smart licensing component of NSO runs its own Java virtual machine. Usually, the default Java options are sufficient: - - ```yang - leaf java-options { - tailf:info "Smart licensing Java VM start options"; - type string; - default "-Xmx64M -Xms16M - -Djava.security.egd=file:/dev/./urandom"; - description - "Options which NCS will use when starting - the Java VM.";} - ``` - - \ - If you, for some reason, need to modify the Java options, remember to include the default values as found in the YANG model. - -### Validation and Troubleshooting - -#### Available `show` and `debug` Commands - -* `show license all`: Displays all information. -* `show license status`: Displays status information. -* `show license summary`: Displays summary. -* `show license tech`: Displays license tech support information. -* `show license usage`: Displays usage information. -* `debug smart_lic all`: All available Smart Licensing debug flags. diff --git a/administration/management/system-management/log-messages-and-formats.md b/administration/management/system-management/log-messages-and-formats.md deleted file mode 100644 index 2435a193..00000000 --- a/administration/management/system-management/log-messages-and-formats.md +++ /dev/null @@ -1,3602 +0,0 @@ -# Log Messages and Formats - - -
- -AAA_LOAD_FAIL - -AAA_LOAD_FAIL - -* **Severity** - `CRIT` -* **Description** - Failed to load the AAA data, it could be that an external db is misbehaving or AAA is mounted/populated badly -* **Format String** - `"Failed to load AAA: ~s"` - -
- - -
- -ABORT_CAND_COMMIT - -ABORT_CAND_COMMIT - -* **Severity** - `INFO` -* **Description** - Aborting candidate commit, request from user, reverting configuration. -* **Format String** - `"Aborting candidate commit, request from user, reverting configuration."` - -
- - -
- -ABORT_CAND_COMMIT_REBOOT - -ABORT_CAND_COMMIT_REBOOT - -* **Severity** - `INFO` -* **Description** - ConfD restarted while having a ongoing candidate commit timer, reverting configuration. -* **Format String** - `"ConfD restarted while having a ongoing candidate commit timer, reverting configuration."` - -
- - -
- -ABORT_CAND_COMMIT_TERM - -ABORT_CAND_COMMIT_TERM - -* **Severity** - `INFO` -* **Description** - Candidate commit session terminated, reverting configuration. -* **Format String** - `"Candidate commit session terminated, reverting configuration."` - -
- - -
- -ABORT_CAND_COMMIT_TIMER - -ABORT_CAND_COMMIT_TIMER - -* **Severity** - `INFO` -* **Description** - Candidate commit timer expired, reverting configuration. -* **Format String** - `"Candidate commit timer expired, reverting configuration."` - -
- - -
- -ACCEPT_FATAL - -ACCEPT_FATAL - -* **Severity** - `CRIT` -* **Description** - ConfD encountered an OS-specific error indicating that networking support is unavailable. -* **Format String** - `"Fatal error for accept() - ~s"` - -
- - -
- -ACCEPT_FDLIMIT - -ACCEPT_FDLIMIT - -* **Severity** - `CRIT` -* **Description** - ConfD failed to accept a connection due to reaching the process or system-wide file descriptor limit. -* **Format String** - `"Out of file descriptors for accept() - ~s limit reached"` - -
- - -
- -AUTH_LOGIN_FAIL - -AUTH_LOGIN_FAIL - -* **Severity** - `INFO` -* **Description** - A user failed to log in to ConfD. -* **Format String** - `"login failed via ~s from ~s with ~s: ~s"` - -
- - -
- -AUTH_LOGIN_SUCCESS - -AUTH_LOGIN_SUCCESS - -* **Severity** - `INFO` -* **Description** - A user logged into ConfD. -* **Format String** - `"logged in to ~s via ~s from ~s with ~s using ~s authentication"` - -
- - -
- -AUTH_LOGOUT - -AUTH_LOGOUT - -* **Severity** - `INFO` -* **Description** - A user was logged out from ConfD. -* **Format String** - `"logged out <~s> user"` - -
- - -
- -BADCONFIG - -BADCONFIG - -* **Severity** - `CRIT` -* **Description** - confd.conf contained bad data. -* **Format String** - `"Bad configuration: ~s:~s: ~s"` - -
- - -
- -BAD_DEPENDENCY - -BAD_DEPENDENCY - -* **Severity** - `ERR` -* **Description** - A dependency was not found -* **Format String** - `"The dependency node '~s' for node '~s' in module '~s' does not exist"` - -
- - -
- -BAD_NS_HASH - -BAD_NS_HASH - -* **Severity** - `CRIT` -* **Description** - Two namespaces have the same hash value. The namespace hashvalue MUST be unique. You can pass the flag --nshash to confdc when linking the .xso files to force another value for the namespace hash. -* **Format String** - `"~s"` - -
- - -
- -BIND_ERR - -BIND_ERR - -* **Severity** - `CRIT` -* **Description** - ConfD failed to bind to one of the internally used listen sockets. -* **Format String** - `"~s"` - -
- - -
- -BRIDGE_DIED - -BRIDGE_DIED - -* **Severity** - `ERR` -* **Description** - ConfD is configured to start the confd_aaa_bridge and the C program died. -* **Format String** - `"confd_aaa_bridge died - ~s"` - -
- - -
- -CANDIDATE_BAD_FILE_FORMAT - -CANDIDATE_BAD_FILE_FORMAT - -* **Severity** - `WARNING` -* **Description** - The candidate database file has a bad format. The candidate database is reset to the empty database. -* **Format String** - `"Bad format found in candidate db file ~s; resetting candidate"` - -
- - -
- -CANDIDATE_CORRUPT_FILE - -CANDIDATE_CORRUPT_FILE - -* **Severity** - `WARNING` -* **Description** - The candidate database file is corrupt and cannot be read. The candidate database is reset to the empty database. -* **Format String** - `"Corrupt candidate db file ~s; resetting candidate"` - -
- - -
- -CAND_COMMIT_ROLLBACK_DONE - -CAND_COMMIT_ROLLBACK_DONE - -* **Severity** - `INFO` -* **Description** - Candidate commit rollback done -* **Format String** - `"Candidate commit rollback done"` - -
- - -
- -CAND_COMMIT_ROLLBACK_FAILURE - -CAND_COMMIT_ROLLBACK_FAILURE - -* **Severity** - `ERR` -* **Description** - Failed to rollback candidate commit -* **Format String** - `"Failed to rollback candidate commit due to: ~s"` - -
- - -
- -CDB_BACKUP - -CDB_BACKUP - -* **Severity** - `INFO` -* **Description** - CDB data backed up after migration to a new storage backend. -* **Format String** - `"CDB: ~s backed up to ~s"` - -
- - -
- -CDB_BOOT_ERR - -CDB_BOOT_ERR - -* **Severity** - `CRIT` -* **Description** - CDB failed to start. Some grave error in the cdb data files prevented CDB from starting - a recovery from backup is necessary. -* **Format String** - `"CDB boot error: ~s"` - -
- - -
- -CDB_CLIENT_TIMEOUT - -CDB_CLIENT_TIMEOUT - -* **Severity** - `ERR` -* **Description** - A CDB client failed to answer within the timeout period. The client will be disconnected. -* **Format String** - `"CDB client (~s) timed out, waiting for ~s"` - -
- - -
- -CDB_CONFIG_LOST - -CDB_CONFIG_LOST - -* **Severity** - `INFO` -* **Description** - CDB found it's data files but no schema file. CDB recovers by starting from an empty database. -* **Format String** - `"CDB: lost config, deleting DB"` - -
- - -
- -CDB_DB_LOST - -CDB_DB_LOST - -* **Severity** - `INFO` -* **Description** - CDB found it's data schema file but not it's data file. CDB recovers by starting from an empty database. -* **Format String** - `"CDB: lost DB, deleting old config"` - -
- - -
- -CDB_FATAL_ERROR - -CDB_FATAL_ERROR - -* **Severity** - `CRIT` -* **Description** - CDB encounterad an unrecoverable error -* **Format String** - `"fatal error in CDB: ~s"` - -
- - -
- -CDB_INIT_LOAD - -CDB_INIT_LOAD - -* **Severity** - `INFO` -* **Description** - CDB is processing an initialization file. -* **Format String** - `"CDB load: processing file: ~s"` - -
- - -
- -CDB_MIGRATE - -CDB_MIGRATE - -* **Severity** - `INFO` -* **Description** - CDB data migration to a new storage backend. -* **Format String** - `"CDB: migrate ~s to ~s"` - -
- - -
- -CDB_OFFLOAD - -CDB_OFFLOAD - -* **Severity** - `DEBUG` -* **Description** - CDB data offload started. -* **Format String** - `"CDB: offload ~s from memory"` - -
- - -
- -CDB_OP_INIT - -CDB_OP_INIT - -* **Severity** - `ERR` -* **Description** - The operational DB was deleted and re-initialized (because of upgrade or corrupt file) -* **Format String** - `"CDB: Operational DB re-initialized"` - -
- - -
- -CDB_STALE_BACKUP - -CDB_STALE_BACKUP - -* **Severity** - `INFO` -* **Description** - CDB backup data left on disk after migration that can be removed to free up disk space. -* **Format String** - `"CDB: ~s backup file(s) occupying ~sMiB, remove to free up disk space: ~s"` - -
- - -
- -CDB_UPGRADE_FAILED - -CDB_UPGRADE_FAILED - -* **Severity** - `ERR` -* **Description** - Automatic CDB upgrade failed. This means that the data model has been changed in a non-supported way. -* **Format String** - `"CDB: Upgrade failed: ~s"` - -
- - -
- -CGI_REQUEST - -CGI_REQUEST - -* **Severity** - `INFO` -* **Description** - CGI script requested. -* **Format String** - `"CGI: '~s' script with method ~s"` - -
- - -
- -CHANGE_USER - -CHANGE_USER - -* **Severity** - `INFO` -* **Description** - A NETCONF request to change user for authorization was succesfully done. -* **Format String** - `"changed user to ~s, groups ~s"` - -
- - -
- -CLI_CMD - -CLI_CMD - -* **Severity** - `INFO` -* **Description** - User executed a CLI command. -* **Format String** - `"CLI '~s'"` - -
- - -
- -CLI_CMD_ABORTED - -CLI_CMD_ABORTED - -* **Severity** - `INFO` -* **Description** - CLI command aborted. -* **Format String** - `"CLI aborted"` - -
- - -
- -CLI_CMD_DONE - -CLI_CMD_DONE - -* **Severity** - `INFO` -* **Description** - CLI command finished successfully. -* **Format String** - `"CLI done"` - -
- - -
- -CLI_DENIED - -CLI_DENIED - -* **Severity** - `INFO` -* **Description** - User was denied to execute a CLI command due to permissions. -* **Format String** - `"CLI denied '~s'"` - -
- - -
- -COMMIT_INFO - -COMMIT_INFO - -* **Severity** - `INFO` -* **Description** - Information about configuration changes committed to the running data store. -* **Format String** - `"commit ~s"` - -
- - -
- -COMMIT_QUEUE_CORRUPT - -COMMIT_QUEUE_CORRUPT - -* **Severity** - `ERR` -* **Description** - Failed to load commit queue. ConfD recovers by starting from an empty commit queue. -* **Format String** - `"Resetting commit queue due do inconsistent or corrupt data."` - -
- - -
- -CONFIG_CHANGE - -CONFIG_CHANGE - -* **Severity** - `INFO` -* **Description** - A change to ConfD configuration has taken place, e.g., by a reload of the configuration file -* **Format String** - `"ConfD configuration change: ~s"` - -
- - -
- -CONFIG_DEPRECATED - -CONFIG_DEPRECATED - -* **Severity** - `WARNING` -* **Description** - confd.conf contains a deprecated value -* **Format String** - `"Config value is deprecated: ~s"` - -
- - -
- -CONFIG_OBSOLETE - -CONFIG_OBSOLETE - -* **Severity** - `WARNING` -* **Description** - confd.conf contains an obsolete value -* **Format String** - `"Config value is obsolete: ~s"` - -
- - -
- -CONFIG_TRANSACTION_LIMIT - -CONFIG_TRANSACTION_LIMIT - -* **Severity** - `INFO` -* **Description** - Configuration transaction limit reached, rejected new transaction request. -* **Format String** - `"Configuration transaction limit of type '~s' reached, rejected new transaction request"` - -
- - -
- -CONSULT_FILE - -CONSULT_FILE - -* **Severity** - `INFO` -* **Description** - ConfD is reading its configuration file. -* **Format String** - `"Consulting daemon configuration file ~s"` - -
- - -
- -CRYPTO_KEYS_FAILED_LOADING - -CRYPTO_KEYS_FAILED_LOADING - -* **Severity** - `INFO` -* **Description** - Crypto keys failed to load because the old active generation is missing in the new configuration. -* **Format String** - `"Cannot reload crypto keys since the old active generation is missing in the new list of keys."` - -
- - -
- -DAEMON_DIED - -DAEMON_DIED - -* **Severity** - `CRIT` -* **Description** - An external database daemon closed its control socket. -* **Format String** - `"Daemon ~s died"` - -
- - -
- -DAEMON_TIMEOUT - -DAEMON_TIMEOUT - -* **Severity** - `CRIT` -* **Description** - An external database daemon did not respond to a query. -* **Format String** - `"Daemon ~s timed out"` - -
- - -
- -DEVEL_AAA - -DEVEL_AAA - -* **Severity** - `INFO` -* **Description** - Developer aaa log message -* **Format String** - `"~s"` - -
- - -
- -DEVEL_CAPI - -DEVEL_CAPI - -* **Severity** - `INFO` -* **Description** - Developer C api log message -* **Format String** - `"~s"` - -
- - -
- -DEVEL_CDB - -DEVEL_CDB - -* **Severity** - `INFO` -* **Description** - Developer CDB log message -* **Format String** - `"~s"` - -
- - -
- -DEVEL_CONFD - -DEVEL_CONFD - -* **Severity** - `INFO` -* **Description** - Developer ConfD log message -* **Format String** - `"~s"` - -
- - -
- -DEVEL_ECONFD - -DEVEL_ECONFD - -* **Severity** - `INFO` -* **Description** - Developer econfd api log message -* **Format String** - `"~s"` - -
- - -
- -DEVEL_SLS - -DEVEL_SLS - -* **Severity** - `INFO` -* **Description** - Developer smartlicensing api log message -* **Format String** - `"~s"` - -
- - -
- -DEVEL_SNMPA - -DEVEL_SNMPA - -* **Severity** - `INFO` -* **Description** - Developer snmp agent log message -* **Format String** - `"~s"` - -
- - -
- -DEVEL_SNMPGW - -DEVEL_SNMPGW - -* **Severity** - `INFO` -* **Description** - Developer snmp GW log message -* **Format String** - `"~s"` - -
- - -
- -DEVEL_WEBUI - -DEVEL_WEBUI - -* **Severity** - `INFO` -* **Description** - Developer webui log message -* **Format String** - `"~s"` - -
- - -
- -DUPLICATE_MODULE_NAME - -DUPLICATE_MODULE_NAME - -* **Severity** - `CRIT` -* **Description** - Duplicate module name found. -* **Format String** - `"The module name '~s' is both defined in '~s' and '~s'."` - -
- - -
- -DUPLICATE_NAMESPACE - -DUPLICATE_NAMESPACE - -* **Severity** - `CRIT` -* **Description** - Duplicate namespace found. -* **Format String** - `"The namespace ~s is defined in both module ~s and ~s."` - -
- - -
- -DUPLICATE_PREFIX - -DUPLICATE_PREFIX - -* **Severity** - `CRIT` -* **Description** - Duplicate prefix found. -* **Format String** - `"The prefix ~s is defined in both ~s and ~s."` - -
- - -
- -ERRLOG_SIZE_CHANGED - -ERRLOG_SIZE_CHANGED - -* **Severity** - `INFO` -* **Description** - Notify change of log size for error log -* **Format String** - `"Changing size of error log (~s) to ~s (was ~s)"` - -
- - -
- -EVENT_SOCKET_TIMEOUT - -EVENT_SOCKET_TIMEOUT - -* **Severity** - `CRIT` -* **Description** - An event notification subscriber did not reply within the configured timeout period -* **Format String** - `"Event notification subscriber with bitmask ~s timed out, waiting for ~s"` - -
- - -
- -EVENT_SOCKET_WRITE_BLOCK - -EVENT_SOCKET_WRITE_BLOCK - -* **Severity** - `CRIT` -* **Description** - Write on an event socket blocked for too long time -* **Format String** - `"~s"` - -
- - -
- -EXEC_WHEN_CIRCULAR_DEPENDENCY - -EXEC_WHEN_CIRCULAR_DEPENDENCY - -* **Severity** - `WARNING` -* **Description** - An error occurred while evaluating a when-expression. -* **Format String** - `"When-expression evaluation error: circular dependency in ~s"` - -
- - -
- -EXTAUTH_BAD_RET - -EXTAUTH_BAD_RET - -* **Severity** - `ERR` -* **Description** - Authentication is external and the external program returned badly formatted data. -* **Format String** - `"External auth program (user=~s) ret bad output: ~s"` - -
- - -
- -EXT_AUTH_2FA - -EXT_AUTH_2FA - -* **Severity** - `INFO` -* **Description** - External challenge sent to a user. -* **Format String** - `"external challenge sent to ~s from ~s with ~s"` - -
- - -
- -EXT_AUTH_2FA_FAIL - -EXT_AUTH_2FA_FAIL - -* **Severity** - `INFO` -* **Description** - External challenge authentication failed for a user. -* **Format String** - `"external challenge authentication failed via ~s from ~s with ~s: ~s"` - -
- - -
- -EXT_AUTH_2FA_SUCCESS - -EXT_AUTH_2FA_SUCCESS - -* **Severity** - `INFO` -* **Description** - An external challenge authenticated user logged in. -* **Format String** - `"external challenge authentication succeeded via ~s from ~s with ~s, member of groups: ~s~s"` - -
- - -
- -EXT_AUTH_FAIL - -EXT_AUTH_FAIL - -* **Severity** - `INFO` -* **Description** - External authentication failed for a user. -* **Format String** - `"external authentication failed via ~s from ~s with ~s: ~s"` - -
- - -
- -EXT_AUTH_SUCCESS - -EXT_AUTH_SUCCESS - -* **Severity** - `INFO` -* **Description** - An externally authenticated user logged in. -* **Format String** - `"external authentication succeeded via ~s from ~s with ~s, member of groups: ~s~s"` - -
- - -
- -EXT_AUTH_TOKEN_FAIL - -EXT_AUTH_TOKEN_FAIL - -* **Severity** - `INFO` -* **Description** - External token authentication failed for a user. -* **Format String** - `"external token authentication failed via ~s from ~s with ~s: ~s"` - -
- - -
- -EXT_AUTH_TOKEN_SUCCESS - -EXT_AUTH_TOKEN_SUCCESS - -* **Severity** - `INFO` -* **Description** - An externally token authenticated user logged in. -* **Format String** - `"external token authentication succeeded via ~s from ~s with ~s, member of groups: ~s~s"` - -
- - -
- -EXT_BIND_ERR - -EXT_BIND_ERR - -* **Severity** - `CRIT` -* **Description** - ConfD failed to bind to one of the externally visible listen sockets. -* **Format String** - `"~s"` - -
- - -
- -FILE_ERROR - -FILE_ERROR - -* **Severity** - `CRIT` -* **Description** - File error -* **Format String** - `"~s: ~s"` - -
- - -
- -FILE_LOAD - -FILE_LOAD - -* **Severity** - `DEBUG` -* **Description** - System loaded a file. -* **Format String** - `"Loaded file ~s"` - -
- - -
- -FILE_LOADING - -FILE_LOADING - -* **Severity** - `DEBUG` -* **Description** - System starts to load a file. -* **Format String** - `"Loading file ~s"` - -
- - -
- -FILE_LOAD_ERR - -FILE_LOAD_ERR - -* **Severity** - `CRIT` -* **Description** - System tried to load a file in its load path and failed. -* **Format String** - `"Failed to load file ~s: ~s"` - -
- - -
- -FXS_MISMATCH - -FXS_MISMATCH - -* **Severity** - `ERR` -* **Description** - A secondary connected to a primary where the fxs files are different -* **Format String** - `"Fxs mismatch, secondary is not allowed"` - -
- - -
- -GROUP_ASSIGN - -GROUP_ASSIGN - -* **Severity** - `INFO` -* **Description** - A user was assigned to a set of groups. -* **Format String** - `"assigned to groups: ~s"` - -
- - -
- -GROUP_NO_ASSIGN - -GROUP_NO_ASSIGN - -* **Severity** - `INFO` -* **Description** - A user was logged in but wasn't assigned to any groups at all. -* **Format String** - `"Not assigned to any groups - all access is denied"` - -
- - -
- -HA_BAD_VSN - -HA_BAD_VSN - -* **Severity** - `ERR` -* **Description** - A secondary connected to a primary with an incompatible HA protocol version -* **Format String** - `"Incompatible HA version (~s, expected ~s), secondary is not allowed"` - -
- - -
- -HA_DUPLICATE_NODEID - -HA_DUPLICATE_NODEID - -* **Severity** - `ERR` -* **Description** - A secondary arrived with a node id which already exists -* **Format String** - `"Nodeid ~s already exists"` - -
- - -
- -HA_FAILED_CONNECT - -HA_FAILED_CONNECT - -* **Severity** - `ERR` -* **Description** - An attempted library become secondary call failed because the secondary couldn't connect to the primary -* **Format String** - `"Failed to connect to primary: ~s"` - -
- - -
- -HA_SECONDARY_KILLED - -HA_SECONDARY_KILLED - -* **Severity** - `ERR` -* **Description** - A secondary node didn't produce its ticks -* **Format String** - `"Secondary ~s killed due to no ticks"` - -
- - -
- -INTERNAL_ERROR - -INTERNAL_ERROR - -* **Severity** - `CRIT` -* **Description** - A ConfD internal error - should be reported to support@tail-f.com. -* **Format String** - `"Internal error: ~s"` - -
- - -
- -IPC_CAPA_DBG_DUMP_DENIED - -IPC_CAPA_DBG_DUMP_DENIED - -* **Severity** - `INFO` -* **Description** - Debug dump denied for user - capability not enabled. -* **Format String** - `"Debug dump denied for user '~s' - capability not enabled."` - -
- - -
- -IPC_CAPA_DBG_DUMP_GRANTED - -IPC_CAPA_DBG_DUMP_GRANTED - -* **Severity** - `INFO` -* **Description** - Debug dump allowed for user. -* **Format String** - `"Debug dump allowed for user '~s'."` - -
- - -
- -JIT_ENABLED - -JIT_ENABLED - -* **Severity** - `INFO` -* **Description** - Show if JIT is enabled. -* **Format String** - `"JIT ~s"` - -
- - -
- -JSONRPC_LOG_MSG - -JSONRPC_LOG_MSG - -* **Severity** - `INFO` -* **Description** - JSON-RPC traffic log message -* **Format String** - `"JSON-RPC traffic log: ~s"` - -
- - -
- -JSONRPC_REQUEST - -JSONRPC_REQUEST - -* **Severity** - `INFO` -* **Description** - JSON-RPC method requested. -* **Format String** - `"JSON-RPC: '~s' with JSON params ~s"` - -
- - -
- -JSONRPC_REQUEST_ABSOLUTE_TIMEOUT - -JSONRPC_REQUEST_ABSOLUTE_TIMEOUT - -* **Severity** - `INFO` -* **Description** - JSON-RPC absolute timeout. -* **Format String** - `"Stopping session due to absolute timeout: ~s"` - -
- - -
- -JSONRPC_REQUEST_IDLE_TIMEOUT - -JSONRPC_REQUEST_IDLE_TIMEOUT - -* **Severity** - `INFO` -* **Description** - JSON-RPC idle timeout. -* **Format String** - `"Stopping session due to idle timeout: ~s"` - -
- - -
- -JSONRPC_WARN_MSG - -JSONRPC_WARN_MSG - -* **Severity** - `WARNING` -* **Description** - JSON-RPC warning message -* **Format String** - `"JSON-RPC warning: ~s"` - -
- - -
- -KICKER_MISSING_SCHEMA - -KICKER_MISSING_SCHEMA - -* **Severity** - `INFO` -* **Description** - Failed to load kicker schema -* **Format String** - `"Failed to load kicker schema"` - -
- - -
- -LIB_BAD_SIZES - -LIB_BAD_SIZES - -* **Severity** - `ERR` -* **Description** - An application connecting to ConfD used a library version that can't handle the depth and number of keys used by the data model. -* **Format String** - `"Got connect from library with insufficient keypath depth/keys support (~s/~s, needs ~s/~s)"` - -
- - -
- -LIB_BAD_VSN - -LIB_BAD_VSN - -* **Severity** - `ERR` -* **Description** - An application connecting to ConfD used a library version that doesn't match the ConfD version (e.g. old version of the client library). -* **Format String** - `"Got library connect from wrong version (~s, expected ~s)"` - -
- - -
- -LIB_NO_ACCESS - -LIB_NO_ACCESS - -* **Severity** - `ERR` -* **Description** - Access check failure occurred when an application connected to ConfD. -* **Format String** - `"Got library connect with failed access check: ~s"` - -
- - -
- -LISTENER_INFO - -LISTENER_INFO - -* **Severity** - `INFO` -* **Description** - ConfD starts or stops to listen for incoming connections. -* **Format String** - `"~s to listen for ~s on ~s:~s"` - -
- - -
- -LOCAL_AUTH_FAIL - -LOCAL_AUTH_FAIL - -* **Severity** - `INFO` -* **Description** - Authentication for a locally configured user failed. -* **Format String** - `"local authentication failed via ~s from ~s with ~s: ~s"` - -
- - -
- -LOCAL_AUTH_FAIL_BADPASS - -LOCAL_AUTH_FAIL_BADPASS - -* **Severity** - `INFO` -* **Description** - Authentication for a locally configured user failed due to providing bad password. -* **Format String** - `"local authentication failed via ~s from ~s with ~s: ~s"` - -
- - -
- -LOCAL_AUTH_FAIL_NOUSER - -LOCAL_AUTH_FAIL_NOUSER - -* **Severity** - `INFO` -* **Description** - Authentication for a locally configured user failed due to user not found. -* **Format String** - `"local authentication failed via ~s from ~s with ~s: ~s"` - -
- - -
- -LOCAL_AUTH_SUCCESS - -LOCAL_AUTH_SUCCESS - -* **Severity** - `INFO` -* **Description** - A locally authenticated user logged in. -* **Format String** - `"local authentication succeeded via ~s from ~s with ~s, member of groups: ~s"` - -
- - -
- -LOCAL_IPC_ACCESS_DENIED - -LOCAL_IPC_ACCESS_DENIED - -* **Severity** - `INFO` -* **Description** - Local IPC access denied for user. -* **Format String** - `"Local IPC access denied for user ~s connecting from ~s"` - -
- - -
- -LOGGING_DEST_CHANGED - -LOGGING_DEST_CHANGED - -* **Severity** - `INFO` -* **Description** - The target logfile will change to another file -* **Format String** - `"Changing destination of ~s log to ~s"` - -
- - -
- -LOGGING_SHUTDOWN - -LOGGING_SHUTDOWN - -* **Severity** - `INFO` -* **Description** - Logging subsystem terminating -* **Format String** - `"Daemon logging terminating, reason: ~s"` - -
- - -
- -LOGGING_STARTED - -LOGGING_STARTED - -* **Severity** - `INFO` -* **Description** - Logging subsystem started -* **Format String** - `"Daemon logging started"` - -
- - -
- -LOGGING_STARTED_TO - -LOGGING_STARTED_TO - -* **Severity** - `INFO` -* **Description** - Write logs for a subsystem to a specific file -* **Format String** - `"Writing ~s log to ~s"` - -
- - -
- -LOGGING_STATUS_CHANGED - -LOGGING_STATUS_CHANGED - -* **Severity** - `INFO` -* **Description** - Notify a change of logging status (enabled/disabled) for a subsystem -* **Format String** - `"~s ~s log"` - -
- - -
- -LOGIN_REJECTED - -LOGIN_REJECTED - -* **Severity** - `INFO` -* **Description** - Authentication for a user was rejected by application callback. -* **Format String** - `"~s"` - -
- - -
- -MAAPI_LOGOUT - -MAAPI_LOGOUT - -* **Severity** - `INFO` -* **Description** - A maapi user was logged out. -* **Format String** - `"Logged out from maapi ctx=~s (~s)"` - -
- - -
- -MAAPI_WRITE_TO_SOCKET_FAIL - -MAAPI_WRITE_TO_SOCKET_FAIL - -* **Severity** - `INFO` -* **Description** - maapi failed to write to a socket. -* **Format String** - `"maapi server failed to write to a socket. Op: ~s Ecode: ~s Error: ~s~s"` - -
- - -
- -MISSING_AES256CFB128_SETTINGS - -MISSING_AES256CFB128_SETTINGS - -* **Severity** - `ERR` -* **Description** - AES256CFB128 keys were not found in confd.conf -* **Format String** - `"AES256CFB128 keys were not found in confd.conf"` - -
- - -
- -MISSING_AESCFB128_SETTINGS - -MISSING_AESCFB128_SETTINGS - -* **Severity** - `ERR` -* **Description** - AESCFB128 keys were not found in confd.conf -* **Format String** - `"AESCFB128 keys were not found in confd.conf"` - -
- - -
- -MISSING_DES3CBC_SETTINGS - -MISSING_DES3CBC_SETTINGS - -* **Severity** - `ERR` -* **Description** - DES3CBC keys were not found in confd.conf -* **Format String** - `"DES3CBC keys were not found in confd.conf"` - -
- - -
- -MISSING_NS - -MISSING_NS - -* **Severity** - `CRIT` -* **Description** - While validating the consistency of the config - a required namespace was missing. -* **Format String** - `"The namespace ~s could not be found in the loadPath."` - -
- - -
- -MISSING_NS2 - -MISSING_NS2 - -* **Severity** - `CRIT` -* **Description** - While validating the consistency of the config - a required namespace was missing. -* **Format String** - `"The namespace ~s (referenced by ~s) could not be found in the loadPath."` - -
- - -
- -MMAP_SCHEMA_FAIL - -MMAP_SCHEMA_FAIL - -* **Severity** - `ERR` -* **Description** - Failed to setup the shared memory schema -* **Format String** - `"Failed to setup the shared memory schema"` - -
- - -
- -NETCONF - -NETCONF - -* **Severity** - `INFO` -* **Description** - NETCONF traffic log message -* **Format String** - `"~s"` - -
- - -
- -NETCONF_HDR_ERR - -NETCONF_HDR_ERR - -* **Severity** - `ERR` -* **Description** - The cleartext header indicating user and groups was badly formatted. -* **Format String** - `"Got bad NETCONF TCP header"` - -
- - -
- -NIF_LOG - -NIF_LOG - -* **Severity** - `INFO` -* **Description** - Log message from NIF code. -* **Format String** - `"~s: ~s"` - -
- - -
- -NOAAA_CLI_LOGIN - -NOAAA_CLI_LOGIN - -* **Severity** - `INFO` -* **Description** - A user used the --noaaa flag to confd_cli -* **Format String** - `"logged in from the CLI with aaa disabled"` - -
- - -
- -NOTIFICATION_REPLAY_STORE_FAILURE - -NOTIFICATION_REPLAY_STORE_FAILURE - -* **Severity** - `CRIT` -* **Description** - A failure occurred in the builtin notification replay store -* **Format String** - `"~s"` - -
- - -
- -NO_CALLPOINT - -NO_CALLPOINT - -* **Severity** - `CRIT` -* **Description** - ConfD tried to populate an XML tree but no code had registered under the relevant callpoint. -* **Format String** - `"no registration found for callpoint ~s of type=~s"` - -
- - -
- -NO_SUCH_IDENTITY - -NO_SUCH_IDENTITY - -* **Severity** - `CRIT` -* **Description** - The fxs file with the base identity is not loaded -* **Format String** - `"The identity ~s in namespace ~s refers to a non-existing base identity ~s in namespace ~s"` - -
- - -
- -NO_SUCH_NS - -NO_SUCH_NS - -* **Severity** - `CRIT` -* **Description** - A nonexistent namespace was referred to. Typically this means that a .fxs was missing from the loadPath. -* **Format String** - `"No such namespace ~s, used by ~s"` - -
- - -
- -NO_SUCH_TYPE - -NO_SUCH_TYPE - -* **Severity** - `CRIT` -* **Description** - A nonexistent type was referred to from a ns. Typically this means that a bad version of an .fxs file was found in the loadPath. -* **Format String** - `"No such simpleType '~s' in ~s, used by ~s"` - -
- - -
- -NS_LOAD_ERR - -NS_LOAD_ERR - -* **Severity** - `CRIT` -* **Description** - System tried to process a loaded namespace and failed. -* **Format String** - `"Failed to process namespace ~s: ~s"` - -
- - -
- -NS_LOAD_ERR2 - -NS_LOAD_ERR2 - -* **Severity** - `CRIT` -* **Description** - System tried to process a loaded namespace and failed. -* **Format String** - `"Failed to process namespaces: ~s"` - -
- - -
- -OPEN_LOGFILE - -OPEN_LOGFILE - -* **Severity** - `INFO` -* **Description** - Indicate target file for certain type of logging -* **Format String** - `"Logging subsystem, opening log file '~s' for ~s"` - -
- - -
- -PAM_AUTH_FAIL - -PAM_AUTH_FAIL - -* **Severity** - `INFO` -* **Description** - A user failed to authenticate through PAM. -* **Format String** - `"PAM authentication failed via ~s from ~s with ~s: phase ~s, ~s"` - -
- - -
- -PAM_AUTH_SUCCESS - -PAM_AUTH_SUCCESS - -* **Severity** - `INFO` -* **Description** - A PAM authenticated user logged in. -* **Format String** - `"pam authentication succeeded via ~s from ~s with ~s"` - -
- - -
- -PHASE0_STARTED - -PHASE0_STARTED - -* **Severity** - `INFO` -* **Description** - ConfD has just started its start phase 0. -* **Format String** - `"ConfD phase0 started"` - -
- - -
- -PHASE1_STARTED - -PHASE1_STARTED - -* **Severity** - `INFO` -* **Description** - ConfD has just started its start phase 1. -* **Format String** - `"ConfD phase1 started"` - -
- - -
- -READ_STATE_FILE_FAILED - -READ_STATE_FILE_FAILED - -* **Severity** - `CRIT` -* **Description** - Reading of a state file failed -* **Format String** - `"Reading state file failed: ~s: ~s (~s)"` - -
- - -
- -RELOAD - -RELOAD - -* **Severity** - `INFO` -* **Description** - Reload of daemon configuration has been initiated. -* **Format String** - `"Reloading daemon configuration."` - -
- - -
- -REOPEN_LOGS - -REOPEN_LOGS - -* **Severity** - `INFO` -* **Description** - Logging subsystem, reopening log files -* **Format String** - `"Logging subsystem, reopening log files"` - -
- - -
- -RESTCONF_REQUEST - -RESTCONF_REQUEST - -* **Severity** - `INFO` -* **Description** - RESTCONF request -* **Format String** - `"RESTCONF: request with ~s: ~s"` - -
- - -
- -RESTCONF_RESPONSE - -RESTCONF_RESPONSE - -* **Severity** - `INFO` -* **Description** - RESTCONF response -* **Format String** - `"RESTCONF: response with ~s: ~s duration ~s us"` - -
- - -
- -REST_AUTH_FAIL - -REST_AUTH_FAIL - -* **Severity** - `INFO` -* **Description** - Rest authentication for a user failed. -* **Format String** - `"rest authentication failed from ~s"` - -
- - -
- -REST_AUTH_SUCCESS - -REST_AUTH_SUCCESS - -* **Severity** - `INFO` -* **Description** - A rest authenticated user logged in. -* **Format String** - `"rest authentication succeeded from ~s , member of groups: ~s"` - -
- - -
- -REST_REQUEST - -REST_REQUEST - -* **Severity** - `INFO` -* **Description** - REST request -* **Format String** - `"REST: request with ~s: ~s"` - -
- - -
- -REST_RESPONSE - -REST_RESPONSE - -* **Severity** - `INFO` -* **Description** - REST response -* **Format String** - `"REST: response with ~s: ~s duration ~s ms"` - -
- - -
- -ROLLBACK_FAIL_CREATE - -ROLLBACK_FAIL_CREATE - -* **Severity** - `ERR` -* **Description** - Error while creating rollback file. -* **Format String** - `"Error while creating rollback file: ~s: ~s"` - -
- - -
- -ROLLBACK_FAIL_DELETE - -ROLLBACK_FAIL_DELETE - -* **Severity** - `ERR` -* **Description** - Failed to delete rollback file. -* **Format String** - `"Failed to delete rollback file ~s: ~s"` - -
- - -
- -ROLLBACK_FAIL_RENAME - -ROLLBACK_FAIL_RENAME - -* **Severity** - `ERR` -* **Description** - Failed to rename rollback file. -* **Format String** - `"Failed to rename rollback file ~s to ~s: ~s"` - -
- - -
- -ROLLBACK_FAIL_REPAIR - -ROLLBACK_FAIL_REPAIR - -* **Severity** - `ERR` -* **Description** - Failed to repair rollback files. -* **Format String** - `"Failed to repair rollback files."` - -
- - -
- -ROLLBACK_REMOVE - -ROLLBACK_REMOVE - -* **Severity** - `INFO` -* **Description** - Found half created rollback0 file - removing and creating new. -* **Format String** - `"Found half created rollback0 file - removing and creating new"` - -
- - -
- -ROLLBACK_REPAIR - -ROLLBACK_REPAIR - -* **Severity** - `INFO` -* **Description** - Found half created rollback0 file - repairing. -* **Format String** - `"Found half created rollback0 file - repairing"` - -
- - -
- -SESSION_CREATE - -SESSION_CREATE - -* **Severity** - `INFO` -* **Description** - A new user session was created -* **Format String** - `"created new session via ~s from ~s with ~s"` - -
- - -
- -SESSION_LIMIT - -SESSION_LIMIT - -* **Severity** - `INFO` -* **Description** - Session limit reached, rejected new session request. -* **Format String** - `"Session limit of type '~s' reached, rejected new session request"` - -
- - -
- -SESSION_MAX_EXCEEDED - -SESSION_MAX_EXCEEDED - -* **Severity** - `INFO` -* **Description** - A user failed to create a new user sessions due to exceeding sessions limits -* **Format String** - `"could not create new session via ~s from ~s with ~s due to session limits"` - -
- - -
- -SESSION_TERMINATION - -SESSION_TERMINATION - -* **Severity** - `INFO` -* **Description** - A user session was terminated due to specified reason -* **Format String** - `"terminated session (reason: ~s)"` - -
- - -
- -SKIP_FILE_LOADING - -SKIP_FILE_LOADING - -* **Severity** - `DEBUG` -* **Description** - System skips a file. -* **Format String** - `"Skipping file ~s: ~s"` - -
- - -
- -SNMP_AUTHENTICATION_FAILED - -SNMP_AUTHENTICATION_FAILED - -* **Severity** - `INFO` -* **Description** - An SNMP authentication failed. -* **Format String** - `"SNMP authentication failed: ~s"` - -
- - -
- -SNMP_CANT_LOAD_MIB - -SNMP_CANT_LOAD_MIB - -* **Severity** - `CRIT` -* **Description** - The SNMP Agent failed to load a MIB file -* **Format String** - `"Can't load MIB file: ~s"` - -
- - -
- -SNMP_MIB_LOADING - -SNMP_MIB_LOADING - -* **Severity** - `DEBUG` -* **Description** - SNMP Agent loading a MIB file -* **Format String** - `"Loading MIB: ~s"` - -
- - -
- -SNMP_NOT_A_TRAP - -SNMP_NOT_A_TRAP - -* **Severity** - `INFO` -* **Description** - An UDP package was received on the trap receiving port, but it's not an SNMP trap. -* **Format String** - `"SNMP gateway: Non-trap received from ~s"` - -
- - -
- -SNMP_READ_STATE_FILE_FAILED - -SNMP_READ_STATE_FILE_FAILED - -* **Severity** - `CRIT` -* **Description** - Read SNMP agent state file failed -* **Format String** - `"Read state file failed: ~s: ~s"` - -
- - -
- -SNMP_REQUIRES_CDB - -SNMP_REQUIRES_CDB - -* **Severity** - `WARNING` -* **Description** - The SNMP agent requires CDB to be enabled in order to be started. -* **Format String** - `"Can't start SNMP. CDB is not enabled"` - -
- - -
- -SNMP_TRAP_NOT_FORWARDED - -SNMP_TRAP_NOT_FORWARDED - -* **Severity** - `INFO` -* **Description** - An SNMP trap was to be forwarded, but couldn't be. -* **Format String** - `"SNMP gateway: Can't forward trap from ~s; ~s"` - -
- - -
- -SNMP_TRAP_NOT_RECOGNIZED - -SNMP_TRAP_NOT_RECOGNIZED - -* **Severity** - `INFO` -* **Description** - An SNMP trap was received on the trap receiving port, but its definition is not known -* **Format String** - `"SNMP gateway: Can't forward trap with OID ~s from ~s; There is no notification with this OID in the loaded models."` - -
- - -
- -SNMP_TRAP_OPEN_PORT - -SNMP_TRAP_OPEN_PORT - -* **Severity** - `ERR` -* **Description** - The port for listening to SNMP traps could not be opened. -* **Format String** - `"SNMP gateway: Can't open trap listening port ~s: ~s"` - -
- - -
- -SNMP_TRAP_UNKNOWN_SENDER - -SNMP_TRAP_UNKNOWN_SENDER - -* **Severity** - `INFO` -* **Description** - An SNMP trap was to be forwarded, but the sender was not listed in confd.conf. -* **Format String** - `"SNMP gateway: Not forwarding trap from ~s; the sender is not recognized"` - -
- - -
- -SNMP_TRAP_V1 - -SNMP_TRAP_V1 - -* **Severity** - `INFO` -* **Description** - An SNMP v1 trap was received on the trap receiving port, but forwarding v1 traps is not supported. -* **Format String** - `"SNMP gateway: V1 trap received from ~s"` - -
- - -
- -SNMP_WRITE_STATE_FILE_FAILED - -SNMP_WRITE_STATE_FILE_FAILED - -* **Severity** - `WARNING` -* **Description** - Write SNMP agent state file failed -* **Format String** - `"Write state file failed: ~s: ~s"` - -
- - -
- -SSH_HOST_KEY_UNAVAILABLE - -SSH_HOST_KEY_UNAVAILABLE - -* **Severity** - `ERR` -* **Description** - No SSH host keys available. -* **Format String** - `"No SSH host keys available"` - -
- - -
- -SSH_SUBSYS_ERR - -SSH_SUBSYS_ERR - -* **Severity** - `INFO` -* **Description** - Typically errors where the client doesn't properly send the \"subsystem\" command. -* **Format String** - `"ssh protocol subsys - ~s"` - -
- - -
- -STARTED - -STARTED - -* **Severity** - `INFO` -* **Description** - ConfD has started. -* **Format String** - `"ConfD started vsn: ~s"` - -
- - -
- -STARTING - -STARTING - -* **Severity** - `INFO` -* **Description** - ConfD is starting. -* **Format String** - `"Starting ConfD vsn: ~s"` - -
- - -
- -STOPPING - -STOPPING - -* **Severity** - `INFO` -* **Description** - ConfD is stopping (due to e.g. confd --stop). -* **Format String** - `"ConfD stopping (~s)"` - -
- - -
- -TOKEN_MISMATCH - -TOKEN_MISMATCH - -* **Severity** - `ERR` -* **Description** - A secondary connected to a primary with a bad auth token -* **Format String** - `"Token mismatch, secondary is not allowed"` - -
- - -
- -UPGRADE_ABORTED - -UPGRADE_ABORTED - -* **Severity** - `INFO` -* **Description** - In-service upgrade was aborted. -* **Format String** - `"Upgrade aborted"` - -
- - -
- -UPGRADE_COMMITTED - -UPGRADE_COMMITTED - -* **Severity** - `INFO` -* **Description** - In-service upgrade was committed. -* **Format String** - `"Upgrade committed"` - -
- - -
- -UPGRADE_INIT_STARTED - -UPGRADE_INIT_STARTED - -* **Severity** - `INFO` -* **Description** - In-service upgrade initialization has started. -* **Format String** - `"Upgrade init started"` - -
- - -
- -UPGRADE_INIT_SUCCEEDED - -UPGRADE_INIT_SUCCEEDED - -* **Severity** - `INFO` -* **Description** - In-service upgrade initialization succeeded. -* **Format String** - `"Upgrade init succeeded"` - -
- - -
- -UPGRADE_PERFORMED - -UPGRADE_PERFORMED - -* **Severity** - `INFO` -* **Description** - In-service upgrade has been performed (not committed yet). -* **Format String** - `"Upgrade performed"` - -
- - -
- -WEBUI_LOG_MSG - -WEBUI_LOG_MSG - -* **Severity** - `INFO` -* **Description** - WebUI access log message -* **Format String** - `"WebUI access log: ~s"` - -
- - -
- -WEB_ACTION - -WEB_ACTION - -* **Severity** - `INFO` -* **Description** - User executed a Web UI action. -* **Format String** - `"WebUI action '~s'"` - -
- - -
- -WEB_CMD - -WEB_CMD - -* **Severity** - `INFO` -* **Description** - User executed a Web UI command. -* **Format String** - `"WebUI cmd '~s'"` - -
- - -
- -WEB_COMMIT - -WEB_COMMIT - -* **Severity** - `INFO` -* **Description** - User performed Web UI commit. -* **Format String** - `"WebUI commit ~s"` - -
- - -
- -WRITE_STATE_FILE_FAILED - -WRITE_STATE_FILE_FAILED - -* **Severity** - `CRIT` -* **Description** - Writing of a state file failed -* **Format String** - `"Writing state file failed: ~s: ~s (~s)"` - -
- - -
- -XPATH_EVAL_ERROR1 - -XPATH_EVAL_ERROR1 - -* **Severity** - `WARNING` -* **Description** - An error occurred while evaluating an XPath expression. -* **Format String** - `"XPath evaluation error: ~s for ~s"` - -
- - -
- -XPATH_EVAL_ERROR2 - -XPATH_EVAL_ERROR2 - -* **Severity** - `WARNING` -* **Description** - An error occurred while evaluating an XPath expression. -* **Format String** - `"XPath evaluation error: '~s' resulted in ~s for ~s"` - -
- - -
- -COMMIT_UN_SYNCED_DEV - -COMMIT_UN_SYNCED_DEV - -* **Severity** - `INFO` -* **Description** - Data was committed toward a device with bad or unknown sync state -* **Format String** - `"Committed data towards device ~s which is out of sync"` - -
- - -
- -NCS_DEVICE_OUT_OF_SYNC - -NCS_DEVICE_OUT_OF_SYNC - -* **Severity** - `INFO` -* **Description** - A check-sync action reported out-of-sync for a device -* **Format String** - `"NCS device-out-of-sync Device '~s' Info '~s'"` - -
- - -
- -NCS_JAVA_VM_FAIL - -NCS_JAVA_VM_FAIL - -* **Severity** - `ERR` -* **Description** - The NCS Java VM failure/timeout -* **Format String** - `"The NCS Java VM ~s"` - -
- - -
- -NCS_JAVA_VM_START - -NCS_JAVA_VM_START - -* **Severity** - `INFO` -* **Description** - Starting the NCS Java VM -* **Format String** - `"Starting the NCS Java VM"` - -
- - -
- -NCS_PACKAGE_AUTH_BAD_RET - -NCS_PACKAGE_AUTH_BAD_RET - -* **Severity** - `ERR` -* **Description** - Package authentication program returned badly formatted data. -* **Format String** - `"package authentication using ~s program ret bad output: ~s"` - -
- - -
- -NCS_PACKAGE_AUTH_FAIL - -NCS_PACKAGE_AUTH_FAIL - -* **Severity** - `INFO` -* **Description** - Package authentication failed. -* **Format String** - `"package authentication using ~s failed via ~s from ~s with ~s: ~s"` - -
- - -
- -NCS_PACKAGE_AUTH_SUCCESS - -NCS_PACKAGE_AUTH_SUCCESS - -* **Severity** - `INFO` -* **Description** - A package authenticated user logged in. -* **Format String** - `"package authentication using ~s succeeded via ~s from ~s with ~s, member of groups: ~s~s"` - -
- - -
- -NCS_PACKAGE_BAD_DEPENDENCY - -NCS_PACKAGE_BAD_DEPENDENCY - -* **Severity** - `CRIT` -* **Description** - Bad NCS package dependency -* **Format String** - `"Failed to load NCS package: ~s; required package ~s of version ~s is not present (found ~s)"` - -
- - -
- -NCS_PACKAGE_BAD_NCS_VERSION - -NCS_PACKAGE_BAD_NCS_VERSION - -* **Severity** - `CRIT` -* **Description** - Bad NCS version for package -* **Format String** - `"Failed to load NCS package: ~s; requires NCS version ~s"` - -
- - -
- -NCS_PACKAGE_CHAL_2FA - -NCS_PACKAGE_CHAL_2FA - -* **Severity** - `INFO` -* **Description** - Package authentication challenge sent to a user. -* **Format String** - `"package authentication challenge sent to ~s from ~s with ~s"` - -
- - -
- -NCS_PACKAGE_CHAL_FAIL - -NCS_PACKAGE_CHAL_FAIL - -* **Severity** - `INFO` -* **Description** - Package authentication challenge failed. -* **Format String** - `"package authentication challenge using ~s failed via ~s from ~s with ~s: ~s"` - -
- - -
- -NCS_PACKAGE_CIRCULAR_DEPENDENCY - -NCS_PACKAGE_CIRCULAR_DEPENDENCY - -* **Severity** - `CRIT` -* **Description** - Circular NCS package dependency -* **Format String** - `"Failed to load NCS package: ~s; circular dependency found"` - -
- - -
- -NCS_PACKAGE_COPYING - -NCS_PACKAGE_COPYING - -* **Severity** - `DEBUG` -* **Description** - A package is copied from the load path to private directory -* **Format String** - `"Copying NCS package from ~s to ~s"` - -
- - -
- -NCS_PACKAGE_DUPLICATE - -NCS_PACKAGE_DUPLICATE - -* **Severity** - `CRIT` -* **Description** - Duplicate package found -* **Format String** - `"Failed to load duplicate NCS package ~s: (~s)"` - -
- - -
- -NCS_PACKAGE_STATUS_CHANGE - -NCS_PACKAGE_STATUS_CHANGE - -* **Severity** - `DEBUG` -* **Description** - Status changed for the given package. -* **Format String** - `"package '~s' status changed to '~s'."` - -
- - -
- -NCS_PACKAGE_SYNTAX_ERROR - -NCS_PACKAGE_SYNTAX_ERROR - -* **Severity** - `CRIT` -* **Description** - Syntax error in package file -* **Format String** - `"Failed to load NCS package: ~s; syntax error in package file"` - -
- - -
- -NCS_PACKAGE_UPGRADE_ABORTED - -NCS_PACKAGE_UPGRADE_ABORTED - -* **Severity** - `CRIT` -* **Description** - The CDB upgrade was aborted implying that CDB is untouched. However the package state is changed -* **Format String** - `"NCS package upgrade failed with reason '~s'"` - -
- - -
- -NCS_PACKAGE_UPGRADE_UNSAFE - -NCS_PACKAGE_UPGRADE_UNSAFE - -* **Severity** - `CRIT` -* **Description** - Package upgrade has been aborted due to warnings. -* **Format String** - `"NCS package upgrade has been aborted due to warnings:\n~s"` - -
- - -
- -NCS_PYTHON_VM_FAIL - -NCS_PYTHON_VM_FAIL - -* **Severity** - `ERR` -* **Description** - The NCS Python VM failure/timeout -* **Format String** - `"The NCS Python VM ~s"` - -
- - -
- -NCS_PYTHON_VM_START - -NCS_PYTHON_VM_START - -* **Severity** - `INFO` -* **Description** - Starting the named NCS Python VM -* **Format String** - `"Starting the NCS Python VM ~s"` - -
- - -
- -NCS_PYTHON_VM_START_UPGRADE - -NCS_PYTHON_VM_START_UPGRADE - -* **Severity** - `INFO` -* **Description** - Starting a Python VM to run upgrade code -* **Format String** - `"Starting upgrade of NCS Python package ~s"` - -
- - -
- -NCS_SERVICE_OUT_OF_SYNC - -NCS_SERVICE_OUT_OF_SYNC - -* **Severity** - `INFO` -* **Description** - A check-sync action reported out-of-sync for a service -* **Format String** - `"NCS service-out-of-sync Service '~s' Info '~s'"` - -
- - -
- -NCS_SET_PLATFORM_DATA_ERROR - -NCS_SET_PLATFORM_DATA_ERROR - -* **Severity** - `ERR` -* **Description** - The device failed to set the platform operational data at connect -* **Format String** - `"NCS Device '~s' failed to set platform data Info '~s'"` - -
- - -
- -NCS_SMART_LICENSING_ENTITLEMENT_NOTIFICATION - -NCS_SMART_LICENSING_ENTITLEMENT_NOTIFICATION - -* **Severity** - `INFO` -* **Description** - Smart Licensing Entitlement Notification -* **Format String** - `"Smart Licensing Entitlement Notification: ~s"` - -
- - -
- -NCS_SMART_LICENSING_EVALUATION_COUNTDOWN - -NCS_SMART_LICENSING_EVALUATION_COUNTDOWN - -* **Severity** - `INFO` -* **Description** - Smart Licensing evaluation time remaining -* **Format String** - `"Smart Licensing evaluation time remaining: ~s"` - -
- - -
- -NCS_SMART_LICENSING_FAIL - -NCS_SMART_LICENSING_FAIL - -* **Severity** - `INFO` -* **Description** - The NCS Smart Licensing Java VM failure/timeout -* **Format String** - `"The NCS Smart Licensing Java VM ~s"` - -
- - -
- -NCS_SMART_LICENSING_GLOBAL_NOTIFICATION - -NCS_SMART_LICENSING_GLOBAL_NOTIFICATION - -* **Severity** - `INFO` -* **Description** - Smart Licensing Global Notification -* **Format String** - `"Smart Licensing Global Notification: ~s"` - -
- - -
- -NCS_SMART_LICENSING_START - -NCS_SMART_LICENSING_START - -* **Severity** - `INFO` -* **Description** - Starting the NCS Smart Licensing Java VM -* **Format String** - `"Starting the NCS Smart Licensing Java VM"` - -
- - -
- -NCS_SNMPM_START - -NCS_SNMPM_START - -* **Severity** - `INFO` -* **Description** - Starting the NCS SNMP manager component -* **Format String** - `"Starting the NCS SNMP manager component"` - -
- - -
- -NCS_SNMPM_STOP - -NCS_SNMPM_STOP - -* **Severity** - `INFO` -* **Description** - The NCS SNMP manager component has been stopped -* **Format String** - `"The NCS SNMP manager component has been stopped"` - -
- - -
- -NCS_SNMP_INIT_ERR - -NCS_SNMP_INIT_ERR - -* **Severity** - `INFO` -* **Description** - Failed to locate snmp_init.xml in loadpath -* **Format String** - `"Failed to locate snmp_init.xml in loadpath ~s"` - -
- - -
- -NCS_UPGRADE_ABORTED_INTERNAL - -NCS_UPGRADE_ABORTED_INTERNAL - -* **Severity** - `CRIT` -* **Description** - The CDB upgrade was aborted due to some internal error. CDB is left untouched -* **Format String** - `"NCS upgrade failed with reason '~s'"` - -
- - -
- -BAD_LOCAL_PASS - -BAD_LOCAL_PASS - -* **Severity** - `INFO` -* **Description** - A locally configured user provided a bad password. -* **Format String** - `"Provided bad password"` - -
- - -
- -EXT_LOGIN - -EXT_LOGIN - -* **Severity** - `INFO` -* **Description** - An externally authenticated user logged in. -* **Format String** - `"Logged in over ~s using externalauth, member of groups: ~s~s"` - -
- - -
- -EXT_NO_LOGIN - -EXT_NO_LOGIN - -* **Severity** - `INFO` -* **Description** - External authentication failed for a user. -* **Format String** - `"failed to login using externalauth: ~s"` - -
- - -
- -NO_SUCH_LOCAL_USER - -NO_SUCH_LOCAL_USER - -* **Severity** - `INFO` -* **Description** - A non existing local user tried to login. -* **Format String** - `"no such local user"` - -
- - -
- -PAM_LOGIN_FAILED - -PAM_LOGIN_FAILED - -* **Severity** - `INFO` -* **Description** - A user failed to login through PAM. -* **Format String** - `"pam phase ~s failed to login through PAM: ~s"` - -
- - -
- -PAM_NO_LOGIN - -PAM_NO_LOGIN - -* **Severity** - `INFO` -* **Description** - A user failed to login through PAM -* **Format String** - `"failed to login through PAM: ~s"` - -
- - -
- -SSH_LOGIN - -SSH_LOGIN - -* **Severity** - `INFO` -* **Description** - A user logged into ConfD's builtin ssh server. -* **Format String** - `"logged in over ssh from ~s with authmeth:~s"` - -
- - -
- -SSH_LOGOUT - -SSH_LOGOUT - -* **Severity** - `INFO` -* **Description** - A user was logged out from ConfD's builtin ssh server. -* **Format String** - `"Logged out ssh <~s> user"` - -
- - -
- -SSH_NO_LOGIN - -SSH_NO_LOGIN - -* **Severity** - `INFO` -* **Description** - A user failed to login to ConfD's builtin SSH server. -* **Format String** - `"Failed to login over ssh: ~s"` - -
- - -
- -WEB_LOGIN - -WEB_LOGIN - -* **Severity** - `INFO` -* **Description** - A user logged in through the WebUI. -* **Format String** - `"logged in through Web UI from ~s"` - -
- - -
- -WEB_LOGOUT - -WEB_LOGOUT - -* **Severity** - `INFO` -* **Description** - A Web UI user logged out. -* **Format String** - `"logged out from Web UI"` - -
- diff --git a/adtran-dpoe/README-ned-settings.md b/adtran-dpoe/README-ned-settings.md new file mode 100644 index 00000000..b9ca7b3c --- /dev/null +++ b/adtran-dpoe/README-ned-settings.md @@ -0,0 +1,614 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/adtran-dpoe/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/adtran-dpoe/ + device + /ncs:/device/devices/device:/ned-settings/adtran-dpoe/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings adtran-dpoe + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings adtran-dpoe + 2. developer + 3. proxy + 4. proxy-2 + 5. connection + 6. transaction + 7. console + 7.1. warning + 7.2. command + 7.3. pattern + 7.4. action + 7.4.1. state + 8. logger + 9. live-status + ``` + + +# 1. ned-settings adtran-dpoe +----------------------------- + + + - extended-parser (default auto) + + Make the cisco-nx NED handle CLI parsing (i.e. transform the running-config from the device to + the model based config tree). + + auto - Uses turbo-mode when available, will use fastest availablemethod to load + data to NSO. + + turbo-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO using maapi + setvalues call. + + turbo-xml-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO in XML + format. + + +# 2. ned-settings adtran-dpoe developer +--------------------------------------- + + Contains settings used by the NED developers. + + + - developer load-from-file + + Make the NED load a file containing raw device config when doing sync-from. Does only work on + NETSIM targets. + + + - developer platform model + + Override device model name/number. + + + - developer platform name + + Override device name. + + + - developer platform version + + Override device version. + + + - developer device-type (default netsim) + + Real or simulated device. + + netsim - netsim. + + device - device. + + + - developer trace-enable (default false) + + Enable developer tracing. WARNING: may choke NSO with large commits|systems. + + + - developer trace-connection (default false) + + Enable connection tracing. WARNING: may choke NSO with IPC messages. + + + - developer trace-timestamp (default false) + + Add timestamp from NED instance in trace messages for debug purpose. + + + - developer progress-verbosity (default verbose) + + Maximum NED verbosity level which will get written in devel.log file. + + disabled - disabled. + + normal - normal. + + verbose - verbose. + + very-verbose - very-verbose. + + debug - debug. + + + - developer load-native-config allow-delete (default false) + + Enable this setting to be able to handle limited delete operations with 'load-native-config'. Please note that + not all syntax available on a real device works, some delete operations can not be parsed by the NED. Use the + 'verbose' flag to 'load-native-config' to see if delete commands can be parsed. Currently this is only supported + when 'extended-parser' is set to 'turbo-xml-mode' + + + - developer load-native-config delete-with-remove (default false) + + Enable this setting to use 'remove' instead of 'delete' when sending delete operations to NSO. This is useful when + doing delete commands for data that might not be present in CDB. Please note that deletes for missing data will still + be part of transaction, and will be sent to device. Use with care, and do proper testing to understand behaviour. + + +# 3. ned-settings adtran-dpoe proxy +----------------------------------- + + Configure NED to access device via a proxy. + + + - proxy remote-connection + + Connection type between ned, proxy and device. + + ssh - Start a new ssh client on proxy and connect to device (i.e. will launch + interactive shell on proxy). + + telnet - Start a new telnet client on proxy and connect to device (i.e. will launch + interactive shell on proxy). + + serial - Connect through a terminal server to device. + + ssh-direct - Direct forward to device using ned local ssh client (i.e. without shell on + proxy). + + telnet-direct - Direct forward to device using ned local telnet client (i.e. without shell on + proxy). + + + - proxy remote-address + + Address of host behind the proxy. + + + - proxy remote-port + + Port of host behind the proxy. + + + - proxy remote-name + + User name on the device behind the proxy. + + + - proxy remote-password + + Password on the device behind the proxy. + + + - proxy remote-secondary-password + + Second password (e.g. enable) on the device behind the proxy .WARNING MUST UPDATE connector + template to use!. + + + - proxy authgroup + + Authentication credentials for the device behind the proxy. + + + - proxy proxy-prompt + + Prompt pattern on the proxy host before connecting to device. + + + - proxy remote-ssh-args + + Additional arguments used to establish proxy connection. + + + - proxy auth-key private-key-file + + Path to openssh formatted private key file for doing public key auth to device behind proxy. + + + - proxy host-key-validation (default false) + + Set this to true to force host-key validation of device behind proxy. + + +# 4. ned-settings adtran-dpoe proxy-2 +------------------------------------- + + Configure NED to access device via a second proxy. + + + - proxy-2 remote-connection + + Connection type between ned, proxy and device. + + ssh - Start a new ssh client on proxy and connect to device (i.e. will launch + interactive shell on proxy). + + telnet - Start a new telnet client on proxy and connect to device (i.e. will launch + interactive shell on proxy). + + serial - Connect through a terminal server to device. + + ssh-direct - Direct forward to device using ned local ssh client (i.e. without shell on + proxy). + + telnet-direct - Direct forward to device using ned local telnet client (i.e. without shell on + proxy). + + + - proxy-2 remote-address + + Address of host behind the proxy. + + + - proxy-2 remote-port + + Port of host behind the proxy. + + + - proxy-2 remote-name + + User name on the device behind the proxy. + + + - proxy-2 remote-password + + Password on the device behind the proxy. + + + - proxy-2 remote-secondary-password + + Second password (e.g. enable) on the device behind the proxy .WARNING MUST UPDATE connector + template to use!. + + + - proxy-2 authgroup + + Authentication credentials for the device behind the proxy. + + + - proxy-2 proxy-prompt + + Prompt pattern on the proxy host before connecting to device. + + + - proxy-2 remote-ssh-args + + Additional arguments used to establish proxy connection. + + + - proxy-2 auth-key private-key-file + + Path to openssh formatted private key file for doing public key auth to device behind proxy. + + + - proxy-2 host-key-validation (default false) + + Set this to true to force host-key validation of device behind proxy. + + +# 5. ned-settings adtran-dpoe connection +---------------------------------------- + + Configure settings specific to the connection between NED and device. + + + - connection ssh client + + Configure the SSH client to use. Relevant only when using the NED with NSO 5.6 or later. + + ganymed - The legacy SSH client. Used on all older versions of NSO. + + sshj - The new SSH client with support for the latest crypto features. This is the default + when using the NED on NSO 5.6 or later. + + + - connection ssh host-key known-hosts-file + + Path to openssh formatted 'known_hosts' file containing valid host keys. + + + - connection ssh host-key public-key-file + + Path to openssh formatted public (.pub) host key file. + + + - connection ssh auth-key private-key-file + + Path to openssh formatted private key file. + + + - connection ssh keep-alive-interval (default 0) + + Configure SSH client keep alive interval in seconds, default 0 (i.e. no keep-alive). The + keep-alive is implemented in the client by sending an ssh 'ignore' message on the given + interval. + + + - connection number-of-retries (default 0) + + Configure max number of retries the NED will try to connect to the device before giving up. + Default 0. + + + - connection time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + + - connection character-set (default UTF-8) + + Character set to use for telnet session. + + + - connection commands meta-data + + Change the default connector. Default 'ned-connector.json'. + + + - connection commands initial-action + + Interactor action used to initialize a connection. + + + - connection commands awaken-console + + Command sent to awaken console during connection. + + + - connection commands send-delay (default 0) + + Delay in ms before sending a command during connection. + + + - connection commands expect-timeout (default 60000) + + Default limit in ms for waiting for command response. + + + - connection logger silent (default false) + + Toggle detailed logs to only written to store. + + +# 6. ned-settings adtran-dpoe transaction +----------------------------------------- + + Transaction specific settings. + + + - transaction trans-id-method (default modeled-config) + + Select the method for calculating transaction-id. + + modeled-config - Use a snapshot of the data of only the modeled parts of running config for + calculation. + + full-config - Use a snapshot of the full running config for calculation. + + device-custom - Use a device custom method to get a value to use for trans-id. + + + - transaction abort-on-diff (default false) + + Enable to detect diff immediately when config is applied (i.e. in commit/abort/revert). + If a diff is detected an exception is thrown, having the effect in commit that the transaction + is aborted (showing the diff). Note that this means some overhead in commit, where whole config + needs to be retrieved from device to do compare. It's a more exact way to detect OOB changes, + silently dropped config, and unknown side-effects to config (i.e. all causing a diff compared + to NSO state). In fact, it's the only method which guarantees that the config was actually + applied as desired. Since this incurs overhead it is strongly adviced that this feature is + only used during development. + + +# 7. ned-settings adtran-dpoe console +------------------------------------- + + Settings used while interacting with a device. + + + - console ignore-errors + + Flag indicating if errors should be ignored. + + + - console ignore-warnings + + Flag indicating if warnings should be ignored. + + + - console ignore-retries + + Flag indicating if retries should be ignored. + + + - console max-retries + + Maximum number of retries of a command. + + + - console retry-delay + + Number of ms before retrying a command. + + + - console send-delay + + Enable delay before sending commands. + + + - console expect-timeout + + Set default timeout for sending commands. + + + - console chunk-size + + Enable executing commands in chunks. + + + - console line-feed + + Overwrites default line-feed character. + + + - console obfuscate-secret + + Secrets will be obfuscated in trace & log files. + + +## 7.1. ned-settings adtran-dpoe console extension warning +---------------------------------------------------------- + + Device warning regex entry list. Use to ignore warnings/errors etc. + + - console extension warning + + - warning + + Warning regular expression, e.g. vlan.* does not exist.*. + + +## 7.2. ned-settings adtran-dpoe console extension command +---------------------------------------------------------- + + Extend available commands to send. + + - console extension command + + - name + + Key id of the command. + + - data + + Command. + + +## 7.3. ned-settings adtran-dpoe console extension pattern +---------------------------------------------------------- + + Extend available patterns to expect. + + - console extension pattern + + - name + + Key id of the pattern. + + - data + + A regular expression. + + +## 7.4. ned-settings adtran-dpoe console extension action +--------------------------------------------------------- + + Extend available actions to perform. + + - console extension action + + - name + + A name for the action. + + - init + + Command sent to intialize action. + + - flush + + Flush device buffer once action is completed. + + +### 7.4.1. ned-settings adtran-dpoe console extension action state +------------------------------------------------------------------ + + Extend state machine with answers/questions to handle. + + - state + + - pattern + + Regular expression indicating action required. + + - method + + Method used to take action. + + reportInfo - reportInfo. + + reportError - reportError. + + reportWarning - reportWarning. + + sendCommand - sendCommand. + + sendSecret - sendSecret. + + sendRetry - sendRetry. + + recoverError - recoverError. + + - argument + + Additional info to method. + + - next (default DONE) + + State once action is taken. + + +# 8. ned-settings adtran-dpoe logger +------------------------------------ + + Settings for controlling logs generated. + + + - logger level (default info) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - logger java (default true) + + Toggle logs to be added to ncs-java-vm.log. + + +# 9. ned-settings adtran-dpoe live-status +----------------------------------------- + + Configure NED settings related to live-status. + + + - live-status time-to-live (default 50) + + Define time-to-live for data fetched from the device via live-status.(default 50). + + diff --git a/adtran-dpoe/README.md b/adtran-dpoe/README.md new file mode 100644 index 00000000..ff2d7165 --- /dev/null +++ b/adtran-dpoe/README.md @@ -0,0 +1,877 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + ``` + + +# 1. General +------------ + + This document describes the adtran-dpoe NED. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | Default emulated device: Adtran DPoE controller v24.1-11846 | + | | | | + | check-sync | yes | Several check-sync strategies acceped (config-hash, last-updated | + | | | timestamp etc) | + | | | | + | partial-sync-from | yes | Based on full-config filtering | + | | | | + | live-status actions | yes | Commands suported as live-status exec: admin|any|debug|ping|show | + | | | | + | live-status show | yes | Supports several TTL-based data commands | + | | | | + | load-native-config | yes | | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Adtran DPoE Controller | 24.1-11846 | NA | Adtran DPoE Controller | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--adtran-dpoe-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-adtran-dpoe-1.0.1.signed.bin + > ./ncs-6.0-adtran-dpoe-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-adtran-dpoe-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-adtran-dpoe-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `adtran-dpoe-.`: + + ``` + > tar xfz ncs-6.0-adtran-dpoe-1.0.1.tar.gz + > ls -d */ + adtran-dpoe-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package adtran-dpoe-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package adtran-dpoe-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-adtran-dpoe-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package adtran-dpoe-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/adtran-dpoe-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-adtran-dpoe-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-adtran-dpoe-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install adtran-dpoe-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-adtran-dpoe-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-adtran-dpoe-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id adtran-dpoe-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-adtran-dpoe-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings adtran-dpoe logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings adtran-dpoe logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.AdtranDpoe \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings adtran-dpoe logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings adtran-dpoe logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + **SSHJ DEBUG LOGGING** + For issues related to the ssh connection it is often useful to enable full logging in the SSHJ ssh client. + This will make SSHJ print additional log entries in `$NSO_RUNDIR/logs/ncs-java-vm.log`: + +``` +admin@ncs(config)# java-vm java-logging logger net.schmizz.sshj level level-all +admin@ncs(config)# commit +``` + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + NONE + + +# 5. Built in live-status actions +--------------------------------- + + NONE + + +# 6. Built in live-status show +------------------------------ + + Adtran-dpoe NED supports the follwing live-status TTL data: + - connection-status (based on 'show connection status') + - cable-modem (based on 'scm summary total') + + Here is an example of usage for connection-status: + ``` + admin@ncs# show devices device netsim-0 live-status connection-status + NODE + ID HOST MANAGEMENT + ------------------------------------------ + 1 2001:1998:202:3029::11 Connected + 2 2001:1998:202:3029::10 Connected + 3 2001:1998:202:2016::13 Connected + + admin@ncs# + ``` + The NED output can be customized to display the result in a more structured way (eg XML or JSON): + ``` + admin@ncs# show devices device netsim-0 live-status connection-status | display xml + + + + netsim-0 + + + + 1 + 2001:1998:202:3029::11 + Connected + + + 2 + 2001:1998:202:3029::10 + Connected + + + 3 + 2001:1998:202:2016::13 + Connected + + + + + + + admin@ncs# + ``` + + Here is an example of usage for cable-modem (json format): + ``` + admin@ncs# show devices device netsim-0 live-status cable-modem | display json + { + "data": { + "tailf-ncs:devices": { + "device": [ + { + "name": "netsim-0", + "live-status": { + "tailf-ned-adtran-dpoe-stats:cable-modem": [ + { + "interface": "C1/1/1", + "total": "5", + "reg": "5", + "unreg": "0", + "offline": "0", + "init-rc": "0", + "init-d": "0", + "init-o": "0", + "init6s": "0", + "init6o": "0", + "description": "EPON Port 1/1/1" + }, + { + "interface": "C1/1/2", + "total": "1", + "reg": "1", + "unreg": "0", + "offline": "0", + "init-rc": "0", + "init-d": "0", + "init-o": "0", + "init6s": "0", + "init6o": "0", + "description": "EPON Port 1/1/2" + }, + { + "interface": "C1/1/3", + "total": "1", + "reg": "1", + "unreg": "0", + "offline": "0", + "init-rc": "0", + "init-d": "0", + "init-o": "0", + "init6s": "0", + "init6o": "0", + "description": "EPON Port 1/1/3" + }, + { + "interface": "C1/1/4", + "total": "0", + "reg": "0", + "unreg": "0", + "offline": "0", + "init-rc": "0", + "init-d": "0", + "init-o": "0", + "init6s": "0", + "init6o": "0", + "description": "EPON Port 1/1/4" + }, + { + "interface": "C2/1/1", + "total": "28", + "reg": "28", + "unreg": "0", + "offline": "0", + "init-rc": "0", + "init-d": "0", + "init-o": "0", + "init6s": "0", + "init6o": "0", + "description": "EPON Port 2/1/1" + }, + { + "interface": "C2/1/2", + "total": "0", + "reg": "0", + "unreg": "0", + "offline": "0", + "init-rc": "0", + "init-d": "0", + "init-o": "0", + "init6s": "0", + "init6o": "0", + "description": "EPON Port 2/1/2" + }, + { + "interface": "C2/1/3", + "total": "0", + "reg": "0", + "unreg": "0", + "offline": "0", + "init-rc": "0", + "init-d": "0", + "init-o": "0", + "init6s": "0", + "init6o": "0", + "description": "EPON Port 2/1/3" + }, + { + "interface": "C2/1/4", + "total": "0", + "reg": "0", + "unreg": "0", + "offline": "0", + "init-rc": "0", + "init-d": "0", + "init-o": "0", + "init6s": "0", + "init6o": "0", + "description": "EPON Port 2/1/4" + } + ] + } + } + ] + } + } + } + admin@ncs# + ``` + + +# 7. Limitations +---------------- + + The current adtran-dpoe NED version does not support configuration data (not requested) + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings adtran-dpoe logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings adtran-dpoe logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` diff --git a/adva-825/README-ned-settings.md b/adva-825/README-ned-settings.md new file mode 100644 index 00000000..ef366c35 --- /dev/null +++ b/adva-825/README-ned-settings.md @@ -0,0 +1,289 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/adva-825/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/adva-825/ + device + /ncs:/device/devices/device:/ned-settings/adva-825/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings adva-825 + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings adva-825 + 2. connection + 3. live-status + 4. write + 4.1. inject-command + 5. behaviour + 5.1. errors + 6. developer + 7. proxy + 8. logger + ``` + + +# 1. ned-settings adva-825 +-------------------------- + + + - adva-825 extended-parser (default auto) + + disabled - Load configuration the standard way. + + turbo-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO using maapi + setvalues call. + + turbo-xml-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO in XML + format. + + robust-mode - Makes the NED filter the configuration so that unmodeled content is removed + before being passed to the NSO CLI-engine. This protects against + configuration ending up at the wrong level when NSO CLI parser fallbacks + (which potentially can cause following config to be skipped). + + auto - Uses turbo-mode when available, will use fastest availablemethod to load + data to NSO. If NSO doesn't support data-loading from CLI NED, robust-mode + is used. + + +# 2. ned-settings adva-825 connection +------------------------------------- + + Configure settings specific to the connection between NED and device. + + + - connection number-of-retries (default 0) + + Configure max number of retries the NED will try to connect to the device before giving up. + Default 0. + + + - connection time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + + - connection ssh client + + Configure the SSH client to use. Relevant only when using the NED with NSO 5.6 or later. + + ganymed - The legacy SSH client. Used on all older versions of NSO. + + sshj - The new SSH client with support for the latest crypto features. This is the default + when using the NED on NSO 5.6 or later. + + +# 3. ned-settings adva-825 live-status +-------------------------------------- + + Configure NED settings related to live-status. + + + - live-status time-to-live (default 50) + + Define time-to-live for data fetched from the device via live-status.(default 50). + + +# 4. ned-settings adva-825 write +-------------------------------- + + Settings used when writing to device. + + +## 4.1. ned-settings adva-825 write inject-command +-------------------------------------------------- + + Inject command (before or after) specified config-line upon commit. + + - write inject-command + + - id + + List id, any string. + + - config-line + + The config line where command should be injected (DOTALL regex). + + - command + + The command to inject after|before config-line. + + - where + + before-each - insert command before each matching config-line. + + before-first - insert command before first matching config-line. + + after-each - insert command after each matching config-line. + + after-last - insert command after last matching config-line. + + +# 5. ned-settings adva-825 behaviour +------------------------------------ + + NED specific behaviours. + + + - behaviour cmd-error-retry cmd-output-max-retries (default 90) + + Max number of retries, when sending command to device. + + + - behaviour cmd-error-retry cmd-output-retry-intervel (default 1) + + Specify retry interval in seconds. + + +## 5.1. ned-settings adva-825 behaviour cmd-error-retry errors +-------------------------------------------------------------- + + Device error/warning regexp entry list. + + - behaviour cmd-error-retry errors + + - error + + Warning/error regular expression, e.g. System is currently busy.*. + + +# 6. ned-settings adva-825 developer +------------------------------------ + + Contains settings used for debugging (intended for NED developers). + + + - developer platform model + + Override device model name/number. + + + - developer platform name + + Override device name. + + + - developer platform version + + Override device version. + + + - developer progress-verbosity (default debug) + + Maximum NED verbosity level which will be reported. + + disabled - disabled. + + normal - normal. + + verbose - verbose. + + very-verbose - very-verbose. + + debug - debug. + + +# 7. ned-settings adva-825 proxy +-------------------------------- + + Configure NED to access device via a proxy. + + + - proxy remote-connection + + Connection type between proxy and device. + + ssh - ssh. + + telnet - telnet. + + serial - serial. + + + - proxy remote-address + + Address of host behind the proxy. + + + - proxy remote-port + + Port of host behind the proxy. + + + - proxy remote-name + + User name on the device behind the proxy. + + + - proxy remote-password + + Password on the device behind the proxy. + + + - proxy authgroup + + Authentication credentials for the device behind the proxy. + + + - proxy proxy-prompt + + Prompt pattern on the proxy host. + + + - proxy remote-ssh-args + + Additional arguments used to establish proxy connection. + + +# 8. ned-settings adva-825 logger +--------------------------------- + + Settings for controlling logs generated. + + + - logger level (default info) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - logger java (default false) + + Toggle logs to be added to ncs-java-vm.log. + + diff --git a/adva-825/README.md b/adva-825/README.md new file mode 100644 index 00000000..474ebddf --- /dev/null +++ b/adva-825/README.md @@ -0,0 +1,5794 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + ``` + + +# 1. General +------------ + + This document describes the adva-825 NED. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | ADVA825 NETSIM | + | | | | + | check-sync | yes | Based on trans-id | + | | | | + | partial-sync-from | yes | Based on filtering the full configuration | + | | | | + | live-status actions | yes | live-status show | + | | | | + | live-status show | no | NED does not support TTL'based data | + | | | | + | load-native-config | yes | | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | FSP150CC-GE114 | 6.1.1-183 | 6.1.1- | Hardware device in SJ lab | + | | | 183 | | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--adva-825-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-adva-825-1.0.1.signed.bin + > ./ncs-6.0-adva-825-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-adva-825-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-adva-825-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `adva-825-.`: + + ``` + > tar xfz ncs-6.0-adva-825-1.0.1.tar.gz + > ls -d */ + adva-825-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package adva-825-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package adva-825-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-adva-825-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package adva-825-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/adva-825-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-adva-825-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-adva-825-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install adva-825-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-adva-825-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-adva-825-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id adva-825-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-adva-825-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings adva-825 logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings adva-825 logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.adva825 \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings adva-825 logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings adva-825 logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + The following CLI commands presents an example of the NED CLI usage: + ``` + adva-825:configure system + prompt "LABBRMCO4P001" + exit + + + adva-825:network-element ne-1 + configure env-alm env_alarm_input-1-1-1 + alm-type miscellaneous + alm-notif-code cr + alm-descr "" + alm-input-mode disabled + alm-hold-off enabled + exit + exit + + adva-825:network-element ne-1 + add eompls-pw 1 raw dynamic 10.36.110.6 network-1-1-1-1 enabled 21-5 110 36-5-255 disabled disabled + configure eompls-pw eompls_pw-1-1 + admin-state management + exit + exit + + + adva-825:network-element ne-1 + configure env-alm env_alarm_input-1-1-2 + alm-type miscellaneous + alm-notif-code cr + alm-descr "" + alm-input-mode disabled + alm-hold-off enabled + exit + exit + + + adva-825:network-element ne-1 + configure env-alm env_alarm_input-1-1-3 + alm-type miscellaneous + alm-notif-code cr + alm-descr "" + alm-input-mode disabled + alm-hold-off enabled + exit + exit + + + adva-825:network-element ne-1 + configure env-alm env_alarm_input-1-1-4 + alm-type miscellaneous + alm-notif-code cr + alm-descr "" + alm-input-mode disabled + alm-hold-off enabled + exit + exit + + + adva-825:configure system + alarm-attributes system prim-ntp-svr-fail nsa mn + alarm-attributes system bkup-ntp-svr-fail nsa mn + alarm-attributes system swdl-filetransfer-ip nsa nr + alarm-attributes system swdl-install-ip nsa nr + alarm-attributes system swdl-activation-ip nsa nr + alarm-attributes system swdl-validation-ip nsa nr + alarm-attributes system db-filetransfer-ip nsa nr + alarm-attributes system snmp-dg-host-unreachable nsa mn + alarm-attributes system snmp-dg-res-busy nsa mn + alarm-attributes system ip-address-conflict sa cr + alarm-attributes system db-downgrade nsa nr + alarm-attributes system filetransfer-ip nsa nr + alarm-attributes system operation-ip nsa nr + alarm-attributes system ltp-fail sa mj + alarm-attributes system ltp-in-progress nsa na + alarm-attributes system ipv6-address-conflict sa cr + alarm-attributes system dataexport-ftp-fail nsa mn + alarm-attributes system traffic-arp-table-full nsa mn + alarm-attributes psu ctneqpt nsa mn + alarm-attributes psu eqpt-flt nsa mn + alarm-attributes psu mismatch nsa mn + alarm-attributes psu eqpt-rmvd nsa mn + alarm-attributes psu pwr-no-input-or-unit-fault nsa mn + alarm-attributes access-port farend-duplex-unknown nsa mn + alarm-attributes access-port farend-duplex-unknown sa mj + alarm-attributes access-port efm-oam-dying-gasp nsa mn + alarm-attributes access-port efm-oam-dying-gasp sa cr + alarm-attributes access-port efm-fail nsa mn + alarm-attributes access-port efm-fail sa mn + alarm-attributes access-port efm-rce nsa mn + alarm-attributes access-port efm-rce sa mn + alarm-attributes access-port efm-rld nsa mn + alarm-attributes access-port efm-rld sa mn + alarm-attributes access-port efm-rls nsa mn + alarm-attributes access-port efm-rls sa mn + alarm-attributes access-port lnkdn-deact nsa mn + alarm-attributes access-port lnkdn-deact sa cr + alarm-attributes access-port lnkdn-unisolated nsa mn + alarm-attributes access-port lnkdn-unisolated sa cr + alarm-attributes access-port lnkdn-cbl-flt nsa mn + alarm-attributes access-port lnkdn-cbl-flt sa cr + alarm-attributes access-port lnkdn-cbl-rmvd nsa mn + alarm-attributes access-port lnkdn-cbl-rmvd sa cr + alarm-attributes access-port lnkdn-autoneg-failed nsa mn + alarm-attributes access-port lnkdn-autoneg-failed sa cr + alarm-attributes access-port lnkdn nsa mn + alarm-attributes access-port lnkdn sa cr + alarm-attributes access-port rfi nsa mn + alarm-attributes access-port rfi sa mn + alarm-attributes access-port rx-jabber nsa mn + alarm-attributes access-port rx-jabber sa mj + alarm-attributes access-port sfp-mismatch nsa mn + alarm-attributes access-port sfp-mismatch sa cr + alarm-attributes access-port sfp-rmvd nsa mn + alarm-attributes access-port sfp-rmvd sa cr + alarm-attributes access-port sfp-tx-flt nsa mn + alarm-attributes access-port sfp-tx-flt sa cr + alarm-attributes access-port rmt-efm-lpbk-fail nsa na + alarm-attributes access-port rmt-efm-lpbk-fail sa mn + alarm-attributes access-port syncref nsa mn + alarm-attributes access-port esmc-fail nsa mn + alarm-attributes access-port ql-mismatch nsa mn + alarm-attributes access-port freq-offset nsa mn + alarm-attributes access-port sync-ql-invalid nsa mn + alarm-attributes access-port bw-exceeds-neg-speed nsa mn + alarm-attributes access-port bw-exceeds-neg-speed sa mj + alarm-attributes access-port sfp-non-qualified nsa nr + alarm-attributes access-port lnkdn-master-slave-cfg nsa mn + alarm-attributes access-port lnkdn-master-slave-cfg sa cr + alarm-attributes access-port syncref-locked-out nsa mn + alarm-attributes access-port syncref-forced-switch nsa mn + alarm-attributes access-port syncref-manual-switch nsa mn + alarm-attributes access-port syncref-wtr nsa mn + alarm-attributes access-port link-control-protocol-fail nsa mn + alarm-attributes access-port link-control-protocol-fail sa cr + alarm-attributes access-port link-control-protocol-loopexit nsa mn + alarm-attributes access-port link-control-protocol-loopexit sa cr + alarm-attributes access-port test-alarm nsa mj + alarm-attributes access-port elmi-seq-no-mismatch nsa mn + alarm-attributes access-port elmi-not-oper nsa mn + alarm-attributes access-port amp-no-peer sa mj + alarm-attributes access-port amp-prov-fail nsa mn + alarm-attributes access-port amp-config-fail sa cr + alarm-attributes access-port lpbk-active nsa na + alarm-attributes access-port lpbk-requested nsa na + alarm-attributes network-port forced nsa na + alarm-attributes network-port lockout nsa na + alarm-attributes network-port farend-duplex-unknown nsa mn + alarm-attributes network-port farend-duplex-unknown sa mj + alarm-attributes network-port efm-oam-dying-gasp nsa mn + alarm-attributes network-port efm-oam-dying-gasp sa cr + alarm-attributes network-port efm-fail nsa mn + alarm-attributes network-port efm-fail sa mn + alarm-attributes network-port efm-rce nsa mn + alarm-attributes network-port efm-rce sa mn + alarm-attributes network-port efm-rld nsa mn + alarm-attributes network-port efm-rld sa mn + alarm-attributes network-port efm-rls nsa mn + alarm-attributes network-port efm-rls sa mn + alarm-attributes network-port lnkdn-deact nsa mn + alarm-attributes network-port lnkdn-deact sa cr + alarm-attributes network-port lnkdn-unisolated nsa mn + alarm-attributes network-port lnkdn-unisolated sa cr + alarm-attributes network-port lnkdn-cbl-flt nsa mn + alarm-attributes network-port lnkdn-cbl-flt sa cr + alarm-attributes network-port lnkdn-cbl-rmvd nsa mn + alarm-attributes network-port lnkdn-cbl-rmvd sa cr + alarm-attributes network-port lnkdn-autoneg-failed nsa mn + alarm-attributes network-port lnkdn-autoneg-failed sa cr + alarm-attributes network-port lnkdn nsa mn + alarm-attributes network-port lnkdn sa cr + alarm-attributes network-port rfi nsa mn + alarm-attributes network-port rfi sa mn + alarm-attributes network-port rx-jabber nsa mn + alarm-attributes network-port rx-jabber sa mj + alarm-attributes network-port sfp-mismatch nsa mn + alarm-attributes network-port sfp-mismatch sa cr + alarm-attributes network-port sfp-rmvd nsa mn + alarm-attributes network-port sfp-rmvd sa cr + alarm-attributes network-port sfp-tx-flt nsa mn + alarm-attributes network-port sfp-tx-flt sa cr + alarm-attributes network-port rmt-efm-lpbk-fail nsa na + alarm-attributes network-port rmt-efm-lpbk-fail sa mn + alarm-attributes network-port syncref nsa mn + alarm-attributes network-port esmc-fail nsa mn + alarm-attributes network-port ql-mismatch nsa mn + alarm-attributes network-port freq-offset nsa mn + alarm-attributes network-port sync-ql-invalid nsa mn + alarm-attributes network-port bw-exceeds-neg-speed nsa mn + alarm-attributes network-port bw-exceeds-neg-speed sa mj + alarm-attributes network-port sfp-non-qualified nsa nr + alarm-attributes network-port lnkdn-master-slave-cfg nsa mn + alarm-attributes network-port lnkdn-master-slave-cfg sa cr + alarm-attributes network-port syncref-locked-out nsa mn + alarm-attributes network-port syncref-forced-switch nsa mn + alarm-attributes network-port syncref-manual-switch nsa mn + alarm-attributes network-port syncref-wtr nsa mn + alarm-attributes network-port link-control-protocol-fail nsa mn + alarm-attributes network-port link-control-protocol-fail sa cr + alarm-attributes network-port link-control-protocol-loopexit nsa mn + alarm-attributes network-port link-control-protocol-loopexit sa cr + alarm-attributes network-port test-alarm nsa mj + alarm-attributes network-port elmi-seq-no-mismatch nsa mn + alarm-attributes network-port elmi-not-oper nsa mn + alarm-attributes network-port amp-no-peer sa mj + alarm-attributes network-port amp-prov-fail nsa mn + alarm-attributes network-port amp-config-fail sa cr + alarm-attributes network-port lpbk-active nsa na + alarm-attributes network-port lpbk-requested nsa na + alarm-attributes cfm-mep xcon-ccm sa mn + alarm-attributes cfm-mep err-ccm sa mn + alarm-attributes cfm-mep rem-ccm sa mn + alarm-attributes cfm-mep mac-status sa mn + alarm-attributes cfm-mep rdi sa mn + alarm-attributes cfm-mep ais sa nr + alarm-attributes cfm-qos-shaper shaper-btd nsa na + alarm-attributes dcn-port lnkdn sa na + alarm-attributes lag lnkdn-deact sa cr + alarm-attributes lag lnkdn sa cr + alarm-attributes lag link-control-protocol-fail sa cr + alarm-attributes lag link-control-protocol-loopexit sa cr + alarm-attributes lag amp-no-peer sa mj + alarm-attributes lag amp-prov-fail nsa mn + alarm-attributes lag amp-config-fail sa cr + alarm-attributes lag lpbk-active nsa na + alarm-attributes lag lpbk-requested nsa na + alarm-attributes mobile-modem lnkdn nsa na + alarm-attributes mobile-modem usbdevicemea nsa na + alarm-attributes mobile-modem usbdevicenonqualified nsa na + alarm-attributes mobile-modem usbdeviceremoved nsa na + alarm-attributes mobile-modem no-sim-card nsa na + alarm-attributes erp-group erp-fop-mismatch sa mj + alarm-attributes erp-group erp-fop-timeout nsa mn + alarm-attributes erp-group erp-blockport0-rpl nsa nr + alarm-attributes erp-group erp-blockport0-sf nsa na + alarm-attributes erp-group erp-blockport0-ms nsa na + alarm-attributes erp-group erp-blockport0-fs nsa na + alarm-attributes erp-group erp-blockport0-wtr nsa na + alarm-attributes erp-group erp-blockport1-rpl nsa nr + alarm-attributes erp-group erp-blockport1-sf nsa na + alarm-attributes erp-group erp-blockport1-ms nsa na + alarm-attributes erp-group erp-blockport1-fs nsa na + alarm-attributes erp-group erp-blockport1-wtr nsa na + alarm-attributes sat-responder-session remote-initiated-sat nsa mn + alarm-attributes traffic-ip-intf ip-address-conflict nsa mj + alarm-attributes traffic-ip-intf traffic-ip-intf-outage sa mj + alarm-attributes vrf ip-address-conflict nsa mj + alarm-attributes vrf no-route-resources sa mj + alarm-attributes shelf-nte114pro_he overtemp nsa mn + alarm-attributes shelf-nte114pro_he undertemp nsa mn + alarm-attributes nte114pro_he ctneqpt nsa mn + alarm-attributes nte114pro_he eqpt-flt sa cr + alarm-attributes nte114pro_he mismatch sa cr + alarm-attributes nte114pro_he eqpt-rmvd sa cr + alarm-attributes nte114pro_he test-alarm nsa mj + alarm-attributes dhcp-relay-agent ip-address-conflict sa mj + alarm-attributes bfd-session bfd-session-down sa mn + alarm-attributes eompls-pw destination-unresolved sa mj + alarm-attributes wifi-dongle usbdevicemea nsa na + alarm-attributes wifi-dongle usbdevicenonqualified nsa na + alarm-attributes wifi-dongle usbdeviceremoved nsa na + syslog-server 1 + configure ip-version ipv4 + configure ipv4-address 3.3.3.3 514 + exit + syslog-server 2 + configure ip-version ipv4 + configure ipv4-address 4.4.4.4 514 + exit + syslog-server 3 + configure ip-version ipv4 + configure ipv6-address 0000:0000:0000:0000:0000:0000:0000:0000 514 + exit + audit-log + syslog-control enabled + log2file-control enabled + exit + security-log + syslog-control enabled + exit + alarm-log + syslog-control enabled + log2file-control enabled + exit + acl-entry acl-1 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-2 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-3 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-4 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + + acl-entry acl-5 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-6 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-7 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-8 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-9 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-10 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-11 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-12 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-13 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-14 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-15 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-16 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-17 + control disabled + + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-18 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-19 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-20 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-21 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-22 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-23 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-24 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + acl-entry acl-25 + control disabled + configure permit ipv4 0.0.0.0 255.255.255.255 + exit + exit + + + adva-825:configure user-security + access-order local + auth-protocol radius + auth-type pap + nas-ip-address 5.5.5.5 + nas-ipv6-address 0000:0000:0000:0000:0000:0000:0000:0000 + security-strength low + accounting disabled + config-ssl-strength low + tacacs-privilege-control disabled + tacacs-user-privilege retrieve + security-trap-type disabled + config-rap 1 + ip-version ipv4 + ip-address 5.5.5.5 + ipv6-address 0000:0000:0000:0000:0000:0000:0000:0000 + port 1816 + accounting-port 1813 + order first + timeout 10 + retries 3 + control enabled + exit + config-rap 2 + ip-version ipv4 + ip-address 6.6.6.6 + ipv6-address 0000:0000:0000:0000:0000:0000:0000:0000 + port 1816 + accounting-port 1813 + order second + timeout 10 + retries 3 + control enabled + exit + config-rap 3 + ip-version ipv4 + control disabled + ip-address 0.0.0.0 + ipv6-address 0000:0000:0000:0000:0000:0000:0000:0000 + port 1812 + accounting-port 1813 + order third + timeout 3 + retries 5 + exit + exit + + + adva-825:configure system + profile + configure sac-profile sat_sac_profile-1 + name "SAT_SAC_Profile_1" + flr 5.000000 + ftd 300 + fdv 300 + exit + exit + exit + + + adva-825:configure system + profile + configure sac-profile sat_sac_profile-2 + name "SAT_SAC_Profile_2" + flr 5.000000 + ftd 300 + fdv 300 + exit + exit + exit + + + adva-825:configure system + profile + configure sac-profile sat_sac_profile-3 + name "SAT_SAC_Profile_3" + flr 5.000000 + ftd 300 + fdv 300 + exit + exit + exit + + + adva-825:configure system + profile + configure sac-profile sat_sac_profile-4 + name "SAT_SAC_Profile_4" + flr 5.000000 + ftd 300 + fdv 300 + exit + exit + exit + + + adva-825:configure snmp + add community "turpitude" readonly + exit + + + adva-825:configure snmp + add community "twtinet94" readonly + exit + + + adva-825:configure snmp + add target-params "traphost" snmpv2c snmpv2c "turpitude" no-auth + exit + + + adva-825:configure snmp + add target-address "primary" "7.7.7.7:162" ipv4 1500 3 "trap" "traphost" enabled + exit + + + adva-825:configure snmp + add target-address "secondary" "8.8.8.8:162" ipv4 1500 3 "trap" "traphost" enabled + exit + + + adva-825:network-element ne-1 + prompt "NE-1" + name "LABBRMCO4P001" + contact "Level 3" + location "" + pm-interval 15min + admin-state in-service + exit + + + adva-825:network-element ne-1 + configure psu psu-1-1-1 + admin-state in-service + alias "" + exit + exit + + + + adva-825:network-element ne-1 + configure psu psu-1-1-2 + admin-state in-service + alias "" + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + admin-state in-service + alias "LABBRMCO4P001" + snmp-dying-gasp enabled + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure dcn + admin-state in-service + alias "" + mdix auto + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + configure flow flow-1-1-1-3-1 + eompls-pw none outer-vlantag none + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + configure flow flow-1-1-1-4-1 + eompls-pw none outer-vlantag none + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + configure flow flow-1-1-1-5-1 + eompls-pw none outer-vlantag none + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + eompls-pw none outer-vlantag prio_map_profile-1 + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + admin-state unassigned + service-type evpl + auto-diagnostic disabled + media fiber auto-1000-full + port-shaped-speed 0 + port-shaping disabled + lpbk + inner-vlan1-control disabled + outer-vlan1-control disabled + inner-vlan2-control disabled + outer-vlan2-control disabled + inner-vlan3-control disabled + outer-vlan3-control disabled + exit + efm-oam + oam-control disabled + oam-local-mode oam-active + exit + llf + llf-control disabled + llf-trigger-event none + + llf-tx-action-type no-action + llf-delay 0 + local-link-id 3 + remote-link-ids none + exit + lpbk + block enabled + dst-mac-control disabled + src-mac-control disabled + outer-vlan1 4094-0 + inner-vlan1 4094-0 + outer-vlan2 4094-1 + inner-vlan2 4094-1 + outer-vlan3 4094-2 + inner-vlan3 4094-2 + lpbk-timer 10 + jdsu-lpbk-control disabled + swap-sada none + exit + alias "marcus test ap" + mtu 9600 + afp all-afp + qinq-ethertype 0x0 + pcp-mode none + rx-dei-action use + rx-dei-tag-type ctag-or-stag + tx-dei-action mark-color + tx-dei-tag-type stag + port-vlan-id 3-0 + n2a-vlan-trunking enabled + a2n-push-port-vid disabled + n2a-pop-port-vid disabled + priority-mapping-profile prio_map_profile-1 + a2n-swap-priority-vid disabled + n2a-swap-priority-vid disabled + swap-priority-vid 1 + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + cpd-filter l2pt-tunnel-mac 01:00:0c:cd:cd:d0 + cpd-filter isl discard + cpd-filter pagp pass-thru + cpd-filter udld pass-thru + cpd-filter cdp pass-thru + cpd-filter vtp pass-thru + cpd-filter dtp pass-thru + cpd-filter pvstp+ pass-thru + cpd-filter uplinkfast pass-thru + cpd-filter vlan-bridge pass-thru + cpd-filter l2pt pass-thru + cpd-filter bpdu pass-thru + cpd-filter pause pass-thru + cpd-filter lacp pass-thru + cpd-filter lacp-marker pass-thru + cpd-filter efm-oam discard + cpd-filter ssm discard + cpd-filter port-authen pass-thru + cpd-filter lan-bridges pass-thru + cpd-filter gmrp pass-thru + cpd-filter gvrp pass-thru + cpd-filter garp pass-thru + cpd-filter elmi pass-thru + cpd-filter 01-80-c2-00-00-00 discard + cpd-filter 01-80-c2-00-00-01 discard + cpd-filter 01-80-c2-00-00-02 discard + cpd-filter 01-80-c2-00-00-03 discard + cpd-filter 01-80-c2-00-00-04 discard + cpd-filter 01-80-c2-00-00-05 discard + cpd-filter 01-80-c2-00-00-06 discard + cpd-filter 01-80-c2-00-00-07 discard + cpd-filter 01-80-c2-00-00-08 discard + cpd-filter 01-80-c2-00-00-09 discard + cpd-filter 01-80-c2-00-00-0a discard + cpd-filter 01-80-c2-00-00-0b discard + cpd-filter 01-80-c2-00-00-0c discard + cpd-filter 01-80-c2-00-00-0d discard + cpd-filter 01-80-c2-00-00-0e discard + cpd-filter 01-80-c2-00-00-0f discard + cpd-filter nearest-lldp discard + cpd-filter non-tpmr-lldp discard + cpd-filter customer-lldp discard + clb 1 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + clb 2 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + clb 3 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + clb 4 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + clb 5 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + pm + set-threshold abrrx 0 15min + set-threshold abrtx 0 15min + set-threshold aufd 0 15min + set-threshold apfd 0 15min + set-threshold esbf 0 15min + set-threshold esbp 0 15min + set-threshold esbs 0 15min + set-threshold esc 0 15min + set-threshold escae 37055 15min + set-threshold esde 37055 15min + set-threshold esf 37055 15min + set-threshold esfs 0 15min + set-threshold esj 0 15min + set-threshold esmf 0 15min + set-threshold esmp 0 15min + set-threshold eso 0 15min + set-threshold esof 0 15min + set-threshold esop 37055 15min + set-threshold esp 0 15min + set-threshold esp64 0 15min + set-threshold esp65 0 15min + set-threshold esp128 0 15min + set-threshold esp256 0 15min + set-threshold esp512 0 15min + set-threshold esp1024 0 15min + set-threshold esp1519 0 15min + set-threshold esuf 0 15min + set-threshold esup 37055 15min + set-threshold l2cpfd 0 15min + set-threshold l2cpfp 0 15min + set-threshold l2ptfe 0 15min + set-threshold l2ptfd 0 15min + set-threshold lbc 0 15min + set-threshold opr -80 15min + set-threshold opr-variance 4 15min + set-threshold opt -80 15min + set-threshold opt-variance 4 15min + set-threshold temp 0 15min + set-threshold uas 10 15min + set-threshold ibrmaxrx 0 15min + set-threshold ibrmaxtx 0 15min + set-threshold ibrminrx 0 15min + set-threshold ibrmintx 0 15min + set-threshold ibrrx 0 15min + set-threshold ibrtx 0 15min + set-threshold acl-not-match 0 15min + set-threshold acl-foward-to-cpu 0 15min + set-threshold dhcp-drop-no-interface 0 15min + set-threshold abrrx 0 1day + set-threshold abrtx 0 1day + set-threshold aufd 0 1day + set-threshold apfd 0 1day + set-threshold esbf 0 1day + set-threshold esbp 0 1day + set-threshold esbs 0 1day + set-threshold esc 0 1day + set-threshold escae 3557280 1day + set-threshold esde 3557280 1day + set-threshold esf 3557280 1day + set-threshold esfs 0 1day + set-threshold esj 0 1day + set-threshold esmf 0 1day + set-threshold esmp 0 1day + set-threshold eso 0 1day + set-threshold esof 0 1day + set-threshold esop 3557280 1day + set-threshold esp 0 1day + set-threshold esp64 0 1day + set-threshold esp65 0 1day + set-threshold esp128 0 1day + set-threshold esp256 0 1day + set-threshold esp512 0 1day + set-threshold esp1024 0 1day + set-threshold esp1519 0 1day + set-threshold esuf 0 1day + set-threshold esup 3557280 1day + set-threshold l2cpfd 0 1day + set-threshold l2cpfp 0 1day + set-threshold l2ptfe 0 1day + set-threshold l2ptfd 0 1day + set-threshold lbc 0 1day + set-threshold opr -80 1day + set-threshold opr-variance 4 1day + set-threshold opt -80 1day + set-threshold opt-variance 4 1day + set-threshold temp 0 1day + set-threshold uas 10 1day + set-threshold ibrmaxrx 0 1day + set-threshold ibrmaxtx 0 1day + set-threshold ibrminrx 0 1day + set-threshold ibrmintx 0 1day + set-threshold ibrrx 0 1day + set-threshold ibrtx 0 1day + set-threshold acl-not-match 0 1day + set-threshold acl-foward-to-cpu 0 1day + set-threshold dhcp-drop-no-interface 0 1day + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + admin-state unassigned + service-type epl + port-shaped-speed 0 + port-shaping disabled + mdix auto + lpbk + inner-vlan1-control disabled + outer-vlan1-control disabled + inner-vlan2-control disabled + outer-vlan2-control disabled + inner-vlan3-control disabled + outer-vlan3-control disabled + exit + efm-oam + oam-control disabled + oam-local-mode oam-active + exit + llf + llf-control disabled + llf-trigger-event none + llf-tx-action-type no-action + llf-delay 0 + local-link-id 4 + remote-link-ids none + exit + lpbk + block enabled + dst-mac-control disabled + src-mac-control disabled + outer-vlan1 4094-0 + inner-vlan1 4094-0 + outer-vlan2 4094-1 + inner-vlan2 4094-1 + outer-vlan3 4094-2 + inner-vlan3 4094-2 + lpbk-timer 10 + jdsu-lpbk-control disabled + swap-sada none + exit + alias "" + auto-diagnostic enabled + mtu 9600 + afp all-afp + rx-pause disabled + tx-pause disabled + qinq-ethertype 0x0 + pcp-mode none + rx-dei-action use + rx-dei-tag-type ctag-or-stag + tx-dei-action mark-color + tx-dei-tag-type stag + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + cpd-filter l2pt-tunnel-mac 01:00:0c:cd:cd:d0 + cpd-filter isl pass-thru + cpd-filter pagp pass-thru + cpd-filter udld pass-thru + cpd-filter cdp pass-thru + cpd-filter vtp pass-thru + cpd-filter dtp pass-thru + cpd-filter pvstp+ pass-thru + cpd-filter uplinkfast pass-thru + cpd-filter vlan-bridge pass-thru + cpd-filter l2pt pass-thru + cpd-filter bpdu pass-thru + cpd-filter pause pass-thru + cpd-filter lacp pass-thru + cpd-filter lacp-marker pass-thru + cpd-filter efm-oam discard + cpd-filter ssm discard + cpd-filter port-authen pass-thru + cpd-filter lan-bridges pass-thru + cpd-filter gmrp pass-thru + cpd-filter gvrp pass-thru + cpd-filter garp pass-thru + cpd-filter elmi pass-thru + cpd-filter 01-80-c2-00-00-00 pass-thru + cpd-filter 01-80-c2-00-00-01 discard + cpd-filter 01-80-c2-00-00-02 discard + cpd-filter 01-80-c2-00-00-03 discard + cpd-filter 01-80-c2-00-00-04 discard + cpd-filter 01-80-c2-00-00-05 discard + cpd-filter 01-80-c2-00-00-06 discard + cpd-filter 01-80-c2-00-00-07 discard + cpd-filter 01-80-c2-00-00-08 discard + cpd-filter 01-80-c2-00-00-09 discard + cpd-filter 01-80-c2-00-00-0a discard + cpd-filter 01-80-c2-00-00-0b pass-thru + cpd-filter 01-80-c2-00-00-0c pass-thru + cpd-filter 01-80-c2-00-00-0d pass-thru + cpd-filter 01-80-c2-00-00-0e discard + cpd-filter 01-80-c2-00-00-0f pass-thru + cpd-filter nearest-lldp discard + cpd-filter non-tpmr-lldp discard + cpd-filter customer-lldp discard + clb 1 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + clb 2 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + clb 3 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + clb 4 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + clb 5 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + pm + set-threshold abrrx 0 15min + set-threshold abrtx 0 15min + set-threshold aufd 0 15min + set-threshold apfd 0 15min + set-threshold esbf 0 15min + set-threshold esbp 0 15min + set-threshold esbs 0 15min + set-threshold esc 0 15min + set-threshold escae 37055 15min + set-threshold esde 37055 15min + set-threshold esf 37055 15min + set-threshold esfs 0 15min + set-threshold esj 0 15min + set-threshold esmf 0 15min + set-threshold esmp 0 15min + set-threshold eso 0 15min + set-threshold esof 0 15min + set-threshold esop 37055 15min + set-threshold esp 0 15min + set-threshold esp64 0 15min + set-threshold esp65 0 15min + set-threshold esp128 0 15min + set-threshold esp256 0 15min + set-threshold esp512 0 15min + set-threshold esp1024 0 15min + set-threshold esp1519 0 15min + set-threshold esuf 0 15min + set-threshold esup 37055 15min + set-threshold l2cpfd 0 15min + set-threshold l2cpfp 0 15min + set-threshold l2ptfe 0 15min + set-threshold l2ptfd 0 15min + set-threshold uas 10 15min + set-threshold ibrmaxrx 0 15min + set-threshold ibrmaxtx 0 15min + set-threshold ibrminrx 0 15min + set-threshold ibrmintx 0 15min + set-threshold ibrrx 0 15min + set-threshold ibrtx 0 15min + set-threshold acl-not-match 0 15min + set-threshold acl-foward-to-cpu 0 15min + set-threshold dhcp-drop-no-interface 0 15min + set-threshold abrrx 0 1day + set-threshold abrtx 0 1day + set-threshold aufd 0 1day + set-threshold apfd 0 1day + set-threshold esbf 0 1day + set-threshold esbp 0 1day + set-threshold esbs 0 1day + set-threshold esc 0 1day + set-threshold escae 3557280 1day + set-threshold esde 3557280 1day + set-threshold esf 3557280 1day + set-threshold esfs 0 1day + set-threshold esj 0 1day + set-threshold esmf 0 1day + set-threshold esmp 0 1day + set-threshold eso 0 1day + set-threshold esof 0 1day + set-threshold esop 3557280 1day + set-threshold esp 0 1day + set-threshold esp64 0 1day + set-threshold esp65 0 1day + set-threshold esp128 0 1day + set-threshold esp256 0 1day + set-threshold esp512 0 1day + set-threshold esp1024 0 1day + set-threshold esp1519 0 1day + set-threshold esuf 0 1day + set-threshold esup 3557280 1day + set-threshold l2cpfd 0 1day + set-threshold l2cpfp 0 1day + set-threshold l2ptfe 0 1day + set-threshold l2ptfd 0 1day + set-threshold uas 10 1day + set-threshold ibrmaxrx 0 1day + set-threshold ibrmaxtx 0 1day + set-threshold ibrminrx 0 1day + set-threshold ibrmintx 0 1day + set-threshold ibrrx 0 1day + set-threshold ibrtx 0 1day + set-threshold acl-not-match 0 1day + set-threshold acl-foward-to-cpu 0 1day + set-threshold dhcp-drop-no-interface 0 1day + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + admin-state unassigned + service-type epl + port-shaped-speed 0 + port-shaping disabled + mdix auto + lpbk + inner-vlan1-control disabled + outer-vlan1-control disabled + inner-vlan2-control disabled + outer-vlan2-control disabled + inner-vlan3-control disabled + outer-vlan3-control disabled + exit + efm-oam + oam-control disabled + oam-local-mode oam-active + exit + llf + llf-control disabled + llf-trigger-event none + llf-tx-action-type no-action + llf-delay 0 + local-link-id 5 + remote-link-ids none + exit + lpbk + block enabled + dst-mac-control disabled + src-mac-control disabled + outer-vlan1 4094-0 + inner-vlan1 4094-0 + outer-vlan2 4094-1 + inner-vlan2 4094-1 + + outer-vlan3 4094-2 + inner-vlan3 4094-2 + lpbk-timer 10 + jdsu-lpbk-control disabled + swap-sada none + exit + alias "" + auto-diagnostic enabled + mtu 9600 + afp all-afp + rx-pause disabled + tx-pause disabled + qinq-ethertype 0x0 + pcp-mode none + rx-dei-action use + rx-dei-tag-type ctag-or-stag + tx-dei-action mark-color + tx-dei-tag-type stag + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + cpd-filter l2pt-tunnel-mac 01:00:0c:cd:cd:d0 + cpd-filter isl pass-thru + cpd-filter pagp pass-thru + cpd-filter udld pass-thru + cpd-filter cdp pass-thru + cpd-filter vtp pass-thru + cpd-filter dtp pass-thru + cpd-filter pvstp+ pass-thru + cpd-filter uplinkfast pass-thru + cpd-filter vlan-bridge pass-thru + cpd-filter l2pt pass-thru + cpd-filter bpdu pass-thru + cpd-filter pause pass-thru + cpd-filter lacp pass-thru + cpd-filter lacp-marker pass-thru + cpd-filter efm-oam discard + cpd-filter ssm discard + cpd-filter port-authen pass-thru + cpd-filter lan-bridges pass-thru + cpd-filter gmrp pass-thru + cpd-filter gvrp pass-thru + cpd-filter garp pass-thru + cpd-filter elmi pass-thru + cpd-filter 01-80-c2-00-00-00 pass-thru + cpd-filter 01-80-c2-00-00-01 discard + cpd-filter 01-80-c2-00-00-02 discard + cpd-filter 01-80-c2-00-00-03 discard + cpd-filter 01-80-c2-00-00-04 discard + cpd-filter 01-80-c2-00-00-05 discard + cpd-filter 01-80-c2-00-00-06 discard + cpd-filter 01-80-c2-00-00-07 discard + cpd-filter 01-80-c2-00-00-08 discard + cpd-filter 01-80-c2-00-00-09 discard + cpd-filter 01-80-c2-00-00-0a discard + cpd-filter 01-80-c2-00-00-0b pass-thru + cpd-filter 01-80-c2-00-00-0c pass-thru + cpd-filter 01-80-c2-00-00-0d pass-thru + cpd-filter 01-80-c2-00-00-0e discard + cpd-filter 01-80-c2-00-00-0f pass-thru + cpd-filter nearest-lldp discard + cpd-filter non-tpmr-lldp discard + cpd-filter customer-lldp discard + clb 1 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + clb 2 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + clb 3 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + clb 4 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + clb 5 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + pm + set-threshold abrrx 0 15min + set-threshold abrtx 0 15min + set-threshold aufd 0 15min + set-threshold apfd 0 15min + set-threshold esbf 0 15min + set-threshold esbp 0 15min + set-threshold esbs 0 15min + set-threshold esc 0 15min + set-threshold escae 37055 15min + set-threshold esde 37055 15min + set-threshold esf 37055 15min + set-threshold esfs 0 15min + set-threshold esj 0 15min + set-threshold esmf 0 15min + set-threshold esmp 0 15min + set-threshold eso 0 15min + set-threshold esof 0 15min + set-threshold esop 37055 15min + set-threshold esp 0 15min + set-threshold esp64 0 15min + set-threshold esp65 0 15min + set-threshold esp128 0 15min + set-threshold esp256 0 15min + set-threshold esp512 0 15min + set-threshold esp1024 0 15min + set-threshold esp1519 0 15min + set-threshold esuf 0 15min + set-threshold esup 37055 15min + set-threshold l2cpfd 0 15min + set-threshold l2cpfp 0 15min + set-threshold l2ptfe 0 15min + set-threshold l2ptfd 0 15min + set-threshold uas 10 15min + set-threshold ibrmaxrx 0 15min + set-threshold ibrmaxtx 0 15min + set-threshold ibrminrx 0 15min + set-threshold ibrmintx 0 15min + set-threshold ibrrx 0 15min + set-threshold ibrtx 0 15min + set-threshold acl-not-match 0 15min + set-threshold acl-foward-to-cpu 0 15min + set-threshold dhcp-drop-no-interface 0 15min + set-threshold abrrx 0 1day + set-threshold abrtx 0 1day + set-threshold aufd 0 1day + set-threshold apfd 0 1day + set-threshold esbf 0 1day + set-threshold esbp 0 1day + set-threshold esbs 0 1day + set-threshold esc 0 1day + set-threshold escae 3557280 1day + set-threshold esde 3557280 1day + set-threshold esf 3557280 1day + set-threshold esfs 0 1day + set-threshold esj 0 1day + set-threshold esmf 0 1day + set-threshold esmp 0 1day + set-threshold eso 0 1day + set-threshold esof 0 1day + set-threshold esop 3557280 1day + set-threshold esp 0 1day + set-threshold esp64 0 1day + set-threshold esp65 0 1day + set-threshold esp128 0 1day + set-threshold esp256 0 1day + set-threshold esp512 0 1day + set-threshold esp1024 0 1day + set-threshold esp1519 0 1day + set-threshold esuf 0 1day + set-threshold esup 3557280 1day + set-threshold l2cpfd 0 1day + set-threshold l2cpfp 0 1day + set-threshold l2ptfe 0 1day + set-threshold l2ptfd 0 1day + set-threshold uas 10 1day + set-threshold ibrmaxrx 0 1day + set-threshold ibrmaxtx 0 1day + set-threshold ibrminrx 0 1day + set-threshold ibrmintx 0 1day + set-threshold ibrrx 0 1day + set-threshold ibrtx 0 1day + set-threshold acl-not-match 0 1day + set-threshold acl-foward-to-cpu 0 1day + set-threshold dhcp-drop-no-interface 0 1day + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + admin-state management + service-type evpl + port-shaped-speed 0 + port-shaping disabled + mdix auto + lpbk + inner-vlan1-control disabled + outer-vlan1-control disabled + inner-vlan2-control disabled + outer-vlan2-control disabled + inner-vlan3-control disabled + outer-vlan3-control disabled + exit + efm-oam + oam-control disabled + oam-local-mode oam-active + exit + llf + llf-control disabled + llf-trigger-event none + llf-tx-action-type no-action + llf-delay 0 + local-link-id 6 + remote-link-ids none + exit + lpbk + block enabled + dst-mac-control disabled + src-mac-control disabled + outer-vlan1 4094-0 + inner-vlan1 4094-0 + outer-vlan2 4094-1 + inner-vlan2 4094-1 + outer-vlan3 4094-2 + inner-vlan3 4094-2 + lpbk-timer 10 + jdsu-lpbk-control disabled + swap-sada none + exit + alias "Y.1731 Test Port" + auto-diagnostic enabled + mtu 9600 + afp tagged-afp + qinq-ethertype 0x0 + pcp-mode none + rx-dei-action use + rx-dei-tag-type ctag-or-stag + tx-dei-action mark-color + tx-dei-tag-type stag + port-vlan-id 6-0 + n2a-vlan-trunking enabled + a2n-push-port-vid enabled + n2a-pop-port-vid disabled + priority-mapping-profile prio_map_profile-1 + a2n-swap-priority-vid disabled + n2a-swap-priority-vid disabled + swap-priority-vid 1 + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + cpd-filter l2pt-tunnel-mac 01:00:0c:cd:cd:d0 + cpd-filter isl pass-thru + cpd-filter pagp pass-thru + cpd-filter udld pass-thru + cpd-filter cdp pass-thru + cpd-filter vtp pass-thru + cpd-filter dtp pass-thru + cpd-filter pvstp+ pass-thru + cpd-filter uplinkfast pass-thru + cpd-filter vlan-bridge pass-thru + cpd-filter l2pt pass-thru + cpd-filter bpdu pass-thru + cpd-filter pause pass-thru + cpd-filter lacp pass-thru + cpd-filter lacp-marker pass-thru + cpd-filter efm-oam discard + cpd-filter ssm discard + cpd-filter port-authen pass-thru + cpd-filter lan-bridges pass-thru + cpd-filter gmrp pass-thru + cpd-filter gvrp pass-thru + cpd-filter garp pass-thru + cpd-filter elmi pass-thru + cpd-filter 01-80-c2-00-00-00 discard + cpd-filter 01-80-c2-00-00-01 discard + cpd-filter 01-80-c2-00-00-02 discard + cpd-filter 01-80-c2-00-00-03 discard + cpd-filter 01-80-c2-00-00-04 discard + cpd-filter 01-80-c2-00-00-05 discard + cpd-filter 01-80-c2-00-00-06 discard + cpd-filter 01-80-c2-00-00-07 discard + cpd-filter 01-80-c2-00-00-08 discard + cpd-filter 01-80-c2-00-00-09 discard + cpd-filter 01-80-c2-00-00-0a discard + cpd-filter 01-80-c2-00-00-0b discard + cpd-filter 01-80-c2-00-00-0c discard + cpd-filter 01-80-c2-00-00-0d discard + cpd-filter 01-80-c2-00-00-0e discard + cpd-filter 01-80-c2-00-00-0f discard + cpd-filter nearest-lldp discard + cpd-filter non-tpmr-lldp discard + cpd-filter customer-lldp discard + clb 1 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + clb 2 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + clb 3 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + clb 4 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + clb 5 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + pm + set-threshold abrrx 0 15min + set-threshold abrtx 0 15min + set-threshold aufd 0 15min + set-threshold apfd 0 15min + set-threshold esbf 0 15min + set-threshold esbp 0 15min + set-threshold esbs 0 15min + set-threshold esc 0 15min + set-threshold escae 37055 15min + set-threshold esde 37055 15min + set-threshold esf 37055 15min + set-threshold esfs 0 15min + set-threshold esj 0 15min + set-threshold esmf 0 15min + set-threshold esmp 0 15min + set-threshold eso 0 15min + set-threshold esof 0 15min + set-threshold esop 37055 15min + set-threshold esp 0 15min + set-threshold esp64 0 15min + set-threshold esp65 0 15min + set-threshold esp128 0 15min + set-threshold esp256 0 15min + set-threshold esp512 0 15min + set-threshold esp1024 0 15min + set-threshold esp1519 0 15min + set-threshold esuf 0 15min + set-threshold esup 37055 15min + set-threshold l2cpfd 0 15min + set-threshold l2cpfp 0 15min + set-threshold l2ptfe 0 15min + set-threshold l2ptfd 0 15min + set-threshold uas 10 15min + set-threshold ibrmaxrx 0 15min + set-threshold ibrmaxtx 0 15min + set-threshold ibrminrx 0 15min + set-threshold ibrmintx 0 15min + set-threshold ibrrx 0 15min + set-threshold ibrtx 0 15min + set-threshold acl-not-match 0 15min + set-threshold acl-foward-to-cpu 0 15min + set-threshold dhcp-drop-no-interface 0 15min + set-threshold abrrx 0 1day + set-threshold abrtx 0 1day + set-threshold aufd 0 1day + set-threshold apfd 0 1day + set-threshold esbf 0 1day + set-threshold esbp 0 1day + set-threshold esbs 0 1day + set-threshold esc 0 1day + set-threshold escae 3557280 1day + set-threshold esde 3557280 1day + set-threshold esf 3557280 1day + set-threshold esfs 0 1day + set-threshold esj 0 1day + set-threshold esmf 0 1day + set-threshold esmp 0 1day + set-threshold eso 0 1day + set-threshold esof 0 1day + set-threshold esop 3557280 1day + set-threshold esp 0 1day + set-threshold esp64 0 1day + set-threshold esp65 0 1day + set-threshold esp128 0 1day + set-threshold esp256 0 1day + set-threshold esp512 0 1day + set-threshold esp1024 0 1day + set-threshold esp1519 0 1day + set-threshold esuf 0 1day + set-threshold esup 3557280 1day + set-threshold l2cpfd 0 1day + set-threshold l2cpfp 0 1day + set-threshold l2ptfe 0 1day + set-threshold l2ptfd 0 1day + set-threshold uas 10 1day + set-threshold ibrmaxrx 0 1day + set-threshold ibrmaxtx 0 1day + set-threshold ibrminrx 0 1day + set-threshold ibrmintx 0 1day + set-threshold ibrrx 0 1day + set-threshold ibrtx 0 1day + set-threshold acl-not-match 0 1day + set-threshold acl-foward-to-cpu 0 1day + set-threshold dhcp-drop-no-interface 0 1day + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-1 + admin-state in-service + auto-diagnostic disabled + media fiber auto-1000-full + port-shaped-speed 0 + port-shaping disabled + lpbk + inner-vlan1-control disabled + outer-vlan1-control disabled + inner-vlan2-control disabled + outer-vlan2-control disabled + inner-vlan3-control disabled + outer-vlan3-control disabled + exit + efm-oam + oam-control disabled + oam-local-mode oam-active + exit + alias "15/KEFN/100100/LVLC" + mtu 9638 + qinq-ethertype 0x0 + priority-mapping-profile prio_map_profile-1 + pcp-mode none + rx-dei-action use + rx-dei-tag-type ctag-or-stag + tx-dei-action mark-color + tx-dei-tag-type stag + eompls-src-ip-address 0.0.0.0 + lpbk + block enabled + dst-mac-control disabled + src-mac-control disabled + outer-vlan1 4094-0 + inner-vlan1 4094-0 + outer-vlan2 4094-1 + inner-vlan2 4094-1 + outer-vlan3 4094-2 + inner-vlan3 4094-2 + lpbk-timer 10 + jdsu-lpbk-control disabled + swap-sada none + exit + llf + llf-control disabled + llf-trigger-event none + llf-tx-action-type no-action + llf-delay 0 + local-link-id 1 + remote-link-ids none + delay-asymmetry 0 + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-1 + cpd-filter isl pass-thru + cpd-filter pagp pass-thru + cpd-filter udld pass-thru + cpd-filter cdp pass-thru + cpd-filter vtp pass-thru + cpd-filter dtp pass-thru + cpd-filter pvstp+ pass-thru + cpd-filter uplinkfast pass-thru + cpd-filter vlan-bridge pass-thru + cpd-filter l2pt pass-thru + cpd-filter bpdu pass-thru + cpd-filter pause pass-thru + cpd-filter lacp pass-thru + cpd-filter lacp-marker pass-thru + cpd-filter efm-oam discard + cpd-filter ssm discard + cpd-filter port-authen pass-thru + cpd-filter lan-bridges pass-thru + cpd-filter gmrp pass-thru + cpd-filter gvrp pass-thru + cpd-filter garp pass-thru + cpd-filter elmi pass-thru + cpd-filter 01-80-c2-00-00-00 discard + cpd-filter 01-80-c2-00-00-01 discard + cpd-filter 01-80-c2-00-00-02 discard + cpd-filter 01-80-c2-00-00-03 discard + cpd-filter 01-80-c2-00-00-04 discard + cpd-filter 01-80-c2-00-00-05 discard + cpd-filter 01-80-c2-00-00-06 discard + cpd-filter 01-80-c2-00-00-07 discard + cpd-filter 01-80-c2-00-00-08 discard + cpd-filter 01-80-c2-00-00-09 discard + cpd-filter 01-80-c2-00-00-0a discard + cpd-filter 01-80-c2-00-00-0b discard + cpd-filter 01-80-c2-00-00-0c discard + cpd-filter 01-80-c2-00-00-0d discard + cpd-filter 01-80-c2-00-00-0e discard + cpd-filter 01-80-c2-00-00-0f discard + cpd-filter nearest-lldp discard + cpd-filter non-tpmr-lldp discard + cpd-filter customer-lldp discard + clb 1 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-1 + clb 2 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-1 + clb 3 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-1 + clb 4 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-1 + clb 5 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-1 + pm + set-threshold abrrx 0 15min + set-threshold abrtx 0 15min + set-threshold esbf 0 15min + set-threshold esbp 0 15min + set-threshold esbs 0 15min + set-threshold esc 0 15min + set-threshold escae 37055 15min + set-threshold esde 37055 15min + set-threshold esf 37055 15min + set-threshold esfs 0 15min + set-threshold esj 0 15min + set-threshold esmf 0 15min + set-threshold esmp 0 15min + set-threshold eso 0 15min + set-threshold esof 0 15min + set-threshold esop 37055 15min + set-threshold esp 0 15min + set-threshold esp64 0 15min + set-threshold esp65 0 15min + set-threshold esp128 0 15min + set-threshold esp256 0 15min + set-threshold esp512 0 15min + set-threshold esp1024 0 15min + set-threshold esp1519 0 15min + set-threshold esuf 0 15min + set-threshold esup 37055 15min + set-threshold l2cpfd 0 15min + set-threshold l2cpfp 0 15min + set-threshold lbc 0 15min + set-threshold opr -80 15min + set-threshold opr-variance 4 15min + set-threshold opt -80 15min + set-threshold opt-variance 4 15min + set-threshold psc 0 15min + set-threshold temp 0 15min + set-threshold uas 10 15min + set-threshold pbbunibdadiscard 0 15min + set-threshold pbbgrpbdadiscard 0 15min + set-threshold ibrmaxrx 0 15min + set-threshold ibrmaxtx 0 15min + set-threshold ibrminrx 0 15min + set-threshold ibrmintx 0 15min + set-threshold ibrrx 0 15min + set-threshold ibrtx 0 15min + set-threshold acl-not-match 0 15min + set-threshold acl-foward-to-cpu 0 15min + set-threshold dhcp-drop-no-interface 0 15min + set-threshold abrrx 0 1day + set-threshold abrtx 0 1day + set-threshold esbf 0 1day + set-threshold esbp 0 1day + set-threshold esbs 0 1day + set-threshold esc 0 1day + set-threshold escae 3557280 1day + set-threshold esde 3557280 1day + set-threshold esf 3557280 1day + set-threshold esfs 0 1day + set-threshold esj 0 1day + set-threshold esmf 0 1day + set-threshold esmp 0 1day + set-threshold eso 0 1day + set-threshold esof 0 1day + set-threshold esop 3557280 1day + set-threshold esp 0 1day + set-threshold esp64 0 1day + set-threshold esp65 0 1day + set-threshold esp128 0 1day + set-threshold esp256 0 1day + set-threshold esp512 0 1day + set-threshold esp1024 0 1day + set-threshold esp1519 0 1day + set-threshold esuf 0 1day + set-threshold esup 3557280 1day + set-threshold l2cpfd 0 1day + set-threshold l2cpfp 0 1day + set-threshold lbc 0 1day + set-threshold opr -80 1day + set-threshold opr-variance 4 1day + set-threshold opt -80 1day + set-threshold opt-variance 4 1day + set-threshold psc 0 1day + set-threshold temp 0 1day + set-threshold uas 10 1day + set-threshold pbbunibdadiscard 0 1day + set-threshold pbbgrpbdadiscard 0 1day + set-threshold ibrmaxrx 0 1day + set-threshold ibrmaxtx 0 1day + set-threshold ibrminrx 0 1day + set-threshold ibrmintx 0 1day + set-threshold ibrrx 0 1day + set-threshold ibrtx 0 1day + set-threshold acl-not-match 0 1day + set-threshold acl-foward-to-cpu 0 1day + set-threshold dhcp-drop-no-interface 0 1day + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-2 + admin-state unassigned + media copper auto + port-shaped-speed 0 + port-shaping disabled + mdix auto + lpbk + inner-vlan1-control disabled + outer-vlan1-control disabled + inner-vlan2-control disabled + outer-vlan2-control disabled + inner-vlan3-control disabled + outer-vlan3-control disabled + exit + efm-oam + oam-control disabled + oam-local-mode oam-active + exit + alias "" + auto-diagnostic enabled + mtu 9638 + qinq-ethertype 0x0 + priority-mapping-profile prio_map_profile-1 + pcp-mode none + rx-dei-action use + rx-dei-tag-type ctag-or-stag + tx-dei-action mark-color + tx-dei-tag-type stag + eompls-src-ip-address 0.0.0.0 + lpbk + block enabled + dst-mac-control disabled + src-mac-control disabled + outer-vlan1 4094-0 + inner-vlan1 4094-0 + outer-vlan2 4094-1 + inner-vlan2 4094-1 + outer-vlan3 4094-2 + inner-vlan3 4094-2 + lpbk-timer 10 + jdsu-lpbk-control disabled + swap-sada none + exit + llf + llf-control disabled + llf-trigger-event none + llf-tx-action-type no-action + llf-delay 0 + local-link-id 2 + remote-link-ids none + delay-asymmetry 0 + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-2 + cpd-filter isl pass-thru + cpd-filter pagp pass-thru + cpd-filter udld pass-thru + cpd-filter cdp pass-thru + cpd-filter vtp pass-thru + cpd-filter dtp pass-thru + cpd-filter pvstp+ pass-thru + cpd-filter uplinkfast pass-thru + cpd-filter vlan-bridge pass-thru + cpd-filter l2pt pass-thru + cpd-filter bpdu pass-thru + cpd-filter pause pass-thru + cpd-filter lacp pass-thru + cpd-filter lacp-marker pass-thru + cpd-filter efm-oam discard + cpd-filter ssm discard + cpd-filter port-authen pass-thru + cpd-filter lan-bridges pass-thru + cpd-filter gmrp pass-thru + cpd-filter gvrp pass-thru + cpd-filter garp pass-thru + cpd-filter elmi pass-thru + cpd-filter 01-80-c2-00-00-00 discard + cpd-filter 01-80-c2-00-00-01 discard + cpd-filter 01-80-c2-00-00-02 discard + cpd-filter 01-80-c2-00-00-03 discard + cpd-filter 01-80-c2-00-00-04 discard + cpd-filter 01-80-c2-00-00-05 discard + cpd-filter 01-80-c2-00-00-06 discard + cpd-filter 01-80-c2-00-00-07 discard + cpd-filter 01-80-c2-00-00-08 discard + cpd-filter 01-80-c2-00-00-09 discard + cpd-filter 01-80-c2-00-00-0a discard + cpd-filter 01-80-c2-00-00-0b discard + cpd-filter 01-80-c2-00-00-0c discard + cpd-filter 01-80-c2-00-00-0d discard + cpd-filter 01-80-c2-00-00-0e discard + cpd-filter 01-80-c2-00-00-0f discard + cpd-filter nearest-lldp discard + cpd-filter non-tpmr-lldp discard + cpd-filter customer-lldp discard + clb 1 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-2 + clb 2 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-2 + clb 3 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-2 + clb 4 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-2 + clb 5 + length 0.10 + description "" + control disabled + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-2 + pm + set-threshold abrrx 0 15min + set-threshold abrtx 0 15min + set-threshold esbf 0 15min + set-threshold esbp 0 15min + set-threshold esbs 0 15min + set-threshold esc 0 15min + set-threshold escae 37055 15min + set-threshold esde 37055 15min + set-threshold esf 37055 15min + set-threshold esfs 0 15min + set-threshold esj 0 15min + set-threshold esmf 0 15min + set-threshold esmp 0 15min + set-threshold eso 0 15min + set-threshold esof 0 15min + set-threshold esop 37055 15min + set-threshold esp 0 15min + set-threshold esp64 0 15min + set-threshold esp65 0 15min + set-threshold esp128 0 15min + set-threshold esp256 0 15min + set-threshold esp512 0 15min + set-threshold esp1024 0 15min + set-threshold esp1519 0 15min + set-threshold esuf 0 15min + set-threshold esup 37055 15min + set-threshold l2cpfd 0 15min + set-threshold l2cpfp 0 15min + set-threshold psc 0 15min + set-threshold uas 10 15min + set-threshold pbbunibdadiscard 0 15min + set-threshold pbbgrpbdadiscard 0 15min + set-threshold ibrmaxrx 0 15min + set-threshold ibrmaxtx 0 15min + set-threshold ibrminrx 0 15min + set-threshold ibrmintx 0 15min + set-threshold ibrrx 0 15min + set-threshold ibrtx 0 15min + set-threshold acl-not-match 0 15min + set-threshold acl-foward-to-cpu 0 15min + set-threshold dhcp-drop-no-interface 0 15min + set-threshold abrrx 0 1day + set-threshold abrtx 0 1day + set-threshold esbf 0 1day + set-threshold esbp 0 1day + set-threshold esbs 0 1day + set-threshold esc 0 1day + set-threshold escae 3557280 1day + set-threshold esde 3557280 1day + set-threshold esf 3557280 1day + set-threshold esfs 0 1day + set-threshold esj 0 1day + set-threshold esmf 0 1day + set-threshold esmp 0 1day + set-threshold eso 0 1day + set-threshold esof 0 1day + set-threshold esop 3557280 1day + set-threshold esp 0 1day + set-threshold esp64 0 1day + set-threshold esp65 0 1day + set-threshold esp128 0 1day + set-threshold esp256 0 1day + set-threshold esp512 0 1day + set-threshold esp1024 0 1day + set-threshold esp1519 0 1day + set-threshold esuf 0 1day + set-threshold esup 3557280 1day + set-threshold l2cpfd 0 1day + set-threshold l2cpfp 0 1day + set-threshold psc 0 1day + set-threshold uas 10 1day + set-threshold pbbunibdadiscard 0 1day + set-threshold pbbgrpbdadiscard 0 1day + set-threshold ibrmaxrx 0 1day + set-threshold ibrmaxtx 0 1day + set-threshold ibrminrx 0 1day + set-threshold ibrmintx 0 1day + set-threshold ibrrx 0 1day + set-threshold ibrtx 0 1day + set-threshold acl-not-match 0 1day + set-threshold acl-foward-to-cpu 0 1day + set-threshold dhcp-drop-no-interface 0 1day + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + elmi + elmi-control disabled + n393 4 + t392 15 + async-status enabled + min-async-status-interval 1 + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + elmi + elmi-control disabled + n393 4 + t392 15 + async-status enabled + min-async-status-interval 1 + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + elmi + elmi-control disabled + n393 4 + t392 15 + async-status enabled + min-async-status-interval 1 + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + elmi + elmi-control disabled + n393 4 + t392 15 + async-status enabled + min-async-status-interval 1 + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-1 + elmi + elmi-control disabled + n393 4 + t392 15 + async-status enabled + min-async-status-interval 1 + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-2 + elmi + elmi-control disabled + n393 4 + t392 15 + async-status enabled + min-async-status-interval 1 + exit + exit + exit + exit + + + adva-825:configure system + ecpa-streams 1 + stream-name "stream-1" + framesize 64 + rate 10048000 + payload-type fixed + dest-mac 00:80:ea:00:00:01 + outer-vlan-control disabled + outer-vlan-tag 4094-1 + outer-vlan-ethertype 0x8100 + inner-vlan1-control disabled + inner-vlan1-tag 4094-1 + inner-vlan1-ethertype 0x8100 + inner-vlan2-control disabled + inner-vlan2-tag 4094-1 + inner-vlan2-ethertype 0x8100 + ip-version ipv4 + source-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + target-addr ipv4 0.0.0.0 + target-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + ip-precedence-mode none 0 + use-port-src-mac enabled + exit + exit + + + adva-825:configure system + ecpa-streams 2 + stream-name "stream-2" + framesize 64 + rate 10048000 + payload-type fixed + dest-mac 00:80:ea:00:00:01 + outer-vlan-control disabled + outer-vlan-tag 4094-1 + outer-vlan-ethertype 0x8100 + inner-vlan1-control disabled + inner-vlan1-tag 4094-1 + inner-vlan1-ethertype 0x8100 + inner-vlan2-control disabled + inner-vlan2-tag 4094-1 + inner-vlan2-ethertype 0x8100 + ip-version ipv4 + source-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + target-addr ipv4 0.0.0.0 + target-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + ip-precedence-mode none 0 + use-port-src-mac enabled + exit + exit + + + adva-825:configure system + ecpa-streams 3 + stream-name "stream-3" + framesize 64 + rate 10048000 + payload-type fixed + dest-mac 00:80:ea:00:00:01 + outer-vlan-control disabled + outer-vlan-tag 4094-1 + outer-vlan-ethertype 0x8100 + inner-vlan1-control disabled + inner-vlan1-tag 4094-1 + inner-vlan1-ethertype 0x8100 + inner-vlan2-control disabled + inner-vlan2-tag 4094-1 + inner-vlan2-ethertype 0x8100 + ip-version ipv4 + source-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + target-addr ipv4 0.0.0.0 + target-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + ip-precedence-mode none 0 + use-port-src-mac enabled + exit + exit + + + adva-825:configure system + ecpa-streams 4 + stream-name "stream-4" + framesize 64 + rate 10048000 + payload-type fixed + dest-mac 00:80:ea:00:00:01 + outer-vlan-control disabled + outer-vlan-tag 4094-1 + outer-vlan-ethertype 0x8100 + inner-vlan1-control disabled + inner-vlan1-tag 4094-1 + inner-vlan1-ethertype 0x8100 + inner-vlan2-control disabled + inner-vlan2-tag 4094-1 + inner-vlan2-ethertype 0x8100 + ip-version ipv4 + source-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + target-addr ipv4 0.0.0.0 + target-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + ip-precedence-mode none 0 + use-port-src-mac enabled + exit + exit + + + adva-825:configure system + ecpa-streams 5 + stream-name "stream-5" + framesize 64 + rate 10048000 + payload-type fixed + dest-mac 00:80:ea:00:00:01 + outer-vlan-control disabled + outer-vlan-tag 4094-1 + outer-vlan-ethertype 0x8100 + inner-vlan1-control disabled + inner-vlan1-tag 4094-1 + inner-vlan1-ethertype 0x8100 + inner-vlan2-control disabled + inner-vlan2-tag 4094-1 + inner-vlan2-ethertype 0x8100 + ip-version ipv4 + source-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + target-addr ipv4 0.0.0.0 + target-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + ip-precedence-mode none 0 + use-port-src-mac enabled + exit + exit + + adva-825:configure system + ecpa-streams 6 + stream-name "stream-6" + framesize 64 + rate 10048000 + payload-type fixed + dest-mac 00:80:ea:00:00:01 + outer-vlan-control disabled + outer-vlan-tag 4094-1 + outer-vlan-ethertype 0x8100 + inner-vlan1-control disabled + inner-vlan1-tag 4094-1 + inner-vlan1-ethertype 0x8100 + inner-vlan2-control disabled + inner-vlan2-tag 4094-1 + inner-vlan2-ethertype 0x8100 + ip-version ipv4 + source-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + target-addr ipv4 0.0.0.0 + target-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + ip-precedence-mode none 0 + use-port-src-mac enabled + exit + exit + + + adva-825:configure system + ecpa-streams 7 + stream-name "stream-7" + framesize 64 + rate 10048000 + payload-type fixed + dest-mac 00:80:ea:00:00:01 + outer-vlan-control disabled + outer-vlan-tag 4094-1 + outer-vlan-ethertype 0x8100 + inner-vlan1-control disabled + inner-vlan1-tag 4094-1 + inner-vlan1-ethertype 0x8100 + inner-vlan2-control disabled + inner-vlan2-tag 4094-1 + inner-vlan2-ethertype 0x8100 + ip-version ipv4 + source-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + target-addr ipv4 0.0.0.0 + target-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + ip-precedence-mode none 0 + use-port-src-mac enabled + exit + exit + + + adva-825:configure system + ecpa-streams 8 + stream-name "stream-8" + framesize 64 + rate 10048000 + payload-type fixed + dest-mac 00:80:ea:00:00:01 + outer-vlan-control disabled + outer-vlan-tag 4094-1 + outer-vlan-ethertype 0x8100 + inner-vlan1-control disabled + inner-vlan1-tag 4094-1 + inner-vlan1-ethertype 0x8100 + inner-vlan2-control disabled + inner-vlan2-tag 4094-1 + inner-vlan2-ethertype 0x8100 + ip-version ipv4 + source-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + target-addr ipv4 0.0.0.0 + target-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + ip-precedence-mode none 0 + use-port-src-mac enabled + exit + exit + + + adva-825:configure system + ecpa-streams 9 + stream-name "stream-9" + framesize 64 + rate 10048000 + payload-type fixed + dest-mac 00:80:ea:00:00:01 + outer-vlan-control disabled + outer-vlan-tag 4094-1 + outer-vlan-ethertype 0x8100 + inner-vlan1-control disabled + inner-vlan1-tag 4094-1 + inner-vlan1-ethertype 0x8100 + inner-vlan2-control disabled + inner-vlan2-tag 4094-1 + inner-vlan2-ethertype 0x8100 + ip-version ipv4 + source-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + target-addr ipv4 0.0.0.0 + target-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + ip-precedence-mode none 0 + use-port-src-mac enabled + exit + exit + + + adva-825:configure system + ecpa-streams 10 + stream-name "stream-10" + framesize 64 + rate 10048000 + payload-type fixed + dest-mac 00:80:ea:00:00:01 + outer-vlan-control disabled + outer-vlan-tag 4094-1 + outer-vlan-ethertype 0x8100 + inner-vlan1-control disabled + inner-vlan1-tag 4094-1 + inner-vlan1-ethertype 0x8100 + inner-vlan2-control disabled + inner-vlan2-tag 4094-1 + inner-vlan2-ethertype 0x8100 + ip-version ipv4 + source-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + target-addr ipv4 0.0.0.0 + target-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + ip-precedence-mode none 0 + use-port-src-mac enabled + exit + exit + + + adva-825:configure system + ecpa-streams 11 + stream-name "stream-11" + framesize 64 + rate 10048000 + payload-type fixed + dest-mac 00:80:ea:00:00:01 + outer-vlan-control disabled + outer-vlan-tag 4094-1 + outer-vlan-ethertype 0x8100 + inner-vlan1-control disabled + inner-vlan1-tag 4094-1 + inner-vlan1-ethertype 0x8100 + inner-vlan2-control disabled + inner-vlan2-tag 4094-1 + inner-vlan2-ethertype 0x8100 + ip-version ipv4 + source-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + target-addr ipv4 0.0.0.0 + target-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + ip-precedence-mode none 0 + use-port-src-mac enabled + exit + exit + + adva-825:configure system + ecpa-streams 12 + stream-name "stream-12" + framesize 64 + rate 10048000 + payload-type fixed + dest-mac 00:80:ea:00:00:01 + outer-vlan-control disabled + outer-vlan-tag 4094-1 + outer-vlan-ethertype 0x8100 + inner-vlan1-control disabled + inner-vlan1-tag 4094-1 + inner-vlan1-ethertype 0x8100 + inner-vlan2-control disabled + inner-vlan2-tag 4094-1 + inner-vlan2-ethertype 0x8100 + ip-version ipv4 + source-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + target-addr ipv4 0.0.0.0 + target-addr ipv6 0000:0000:0000:0000:0000:0000:0000:0000 + ip-precedence-mode none 0 + use-port-src-mac enabled + exit + exit + + adva-825:network-element ne-1 + configure tm-params bwp-mode line-rate + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + ecpa-ctrl + src-port access-1-1-1-3 + inject-direction a2n + monitor-direction n2a + test-type duration + test-duration 00:00:00:10 + test-num-frames 1 + test-stream-a 0 + test-stream-b 0 + test-stream-c 0 + exit + exit + exit + + + adva-825:configure communication + add mgmttunnel 1 "ADVA" network-1-1-1-1 ethernet vlan-based ipv4-only enabled 505 disabled 64000 768000 + configure mgmttnl mgmt_tnl-1 + dhcp-control disabled 10.241.47.23 255.255.255.240 + exit + configure mgmttnl mgmt_tnl-1 + buffer-size 32 + cos 7 + exit + configure mgmttnl mgmt_tnl-1 + rip2Pkts-control disabled + dhcp-client-id-control disabled + dhcp-class-id-control enabled + dhcp-host-name-control enabled + dhcp-log-server-control disabled + dhcp-ntp-server-control disabled + dhcp-client-id-type mac-addr + dhcp-host-name-type system-name + exit + exit + + adva-825:configure communication + configure eth0 ip-mode ipv4-only + configure eth0 dhcp-control disabled 1.1.1.115 255.255.255.0 + configure eth0 dhcp-role dhcp-client + configure eth0 dhcp-client-id-type mac-addr + configure eth0 dhcp-class-id-control enabled + configure eth0 dhcp-host-name-control enabled + configure eth0 dhcp-host-name-type system-name + configure eth0 dhcp-log-server-control disabled + configure eth0 dhcp-ntp-server-control disabled + configure eth0 rip2Pkts-control disabled + exit + + adva-825:configure communication + configure src-addr sys-ip-addr "ADVA" "ADVA" + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + add flow flow-1-1-1-3-1 "marcus test flow 3-1" regular-evc enabled disabled disabled disabled 0 disabled push 100-5 disabled none "0:4095" 256000 194304000 eompls-pw none none access-interface access-1-1-1-3 network-interface network-1-1-1-1 flow-based + configure flow flow-1-1-1-3-1 + maximum-flow-bandwidth 6400 + guaranteed-flow-bandwidth 0 + es-frame-loss-threshold 1 + ses-frame-loss-threshold-ratio 30 + policing enabled + policing-control a2n-n2a + n2a-outertag-prio-ctrl disabled + access-learning-ctrl none + network-learning-ctrl none + access-max-forwarding-entries 4096 + network-max-forwarding-entries 4096 + protect-access-learning none + protect-network-learning none + aging-timer 300 + table-full-action forward + n2n-forwarding disabled + a2n-multicast-rate-limit-ctrl disabled + a2n-broadcast-rate-limit-ctrl disabled + a2n-combined-rate-limit-ctrl disabled + cpd-filter isl pass-thru + cpd-filter pagp pass-thru + cpd-filter udld pass-thru + cpd-filter cdp pass-thru + cpd-filter vtp pass-thru + cpd-filter dtp pass-thru + cpd-filter pvstp+ pass-thru + cpd-filter uplinkfast pass-thru + cpd-filter vlan-bridge pass-thru + cpd-filter l2pt pass-thru + cpd-filter bpdu pass-thru + cpd-filter pause pass-thru + cpd-filter lacp pass-thru + cpd-filter lacp-marker pass-thru + cpd-filter efm-oam pass-thru + cpd-filter ssm discard + cpd-filter port-authen pass-thru + cpd-filter lan-bridges pass-thru + cpd-filter gmrp pass-thru + cpd-filter gvrp pass-thru + cpd-filter garp pass-thru + cpd-filter elmi pass-thru + cpd-filter 01-80-c2-00-00-00 discard + cpd-filter 01-80-c2-00-00-01 discard + cpd-filter 01-80-c2-00-00-02 discard + cpd-filter 01-80-c2-00-00-03 discard + cpd-filter 01-80-c2-00-00-04 discard + cpd-filter 01-80-c2-00-00-05 discard + cpd-filter 01-80-c2-00-00-06 discard + cpd-filter 01-80-c2-00-00-07 discard + cpd-filter 01-80-c2-00-00-08 discard + cpd-filter 01-80-c2-00-00-09 discard + cpd-filter 01-80-c2-00-00-0a discard + cpd-filter 01-80-c2-00-00-0b discard + cpd-filter 01-80-c2-00-00-0c discard + cpd-filter 01-80-c2-00-00-0d discard + cpd-filter 01-80-c2-00-00-0e discard + cpd-filter 01-80-c2-00-00-0f discard + cpd-filter nearest-lldp discard + cpd-filter non-tpmr-lldp discard + cpd-filter customer-lldp discard + pm + set-threshold abra2n 0 15min + set-threshold abrrla2n 0 15min + set-threshold abrn2a 0 15min + set-threshold abrrln2a 0 15min + set-threshold l2cpfd 0 15min + set-threshold bytesina2n 0 15min + set-threshold bytesinn2a 0 15min + set-threshold bytesouta2n 0 15min + set-threshold bytesoutn2a 0 15min + set-threshold fmga2n 0 15min + set-threshold fmgn2a 0 15min + set-threshold fmrda2n 0 15min + set-threshold fmrdn2a 0 15min + set-threshold fmya2n 0 15min + set-threshold fmyn2a 0 15min + set-threshold ftda2n 0 15min + set-threshold ses 0 15min + set-threshold uas 10 15min + set-threshold fdhairpin 0 15min + set-threshold fdstaticblock 0 15min + set-threshold learntblentrydiscards 0 15min + set-threshold numlearntblflushes 0 15min + set-threshold ibra2nmax 0 15min + set-threshold ibrrla2nmax 0 15min + set-threshold ibra2nmin 0 15min + set-threshold ibrrla2nmin 0 15min + set-threshold ibra2n 0 15min + set-threshold ibrrla2n 0 15min + set-threshold ibrn2amax 0 15min + set-threshold ibrrln2amax 0 15min + set-threshold ibrn2amin 0 15min + set-threshold ibrrln2amin 0 15min + set-threshold ibrn2a 0 15min + set-threshold ibrrln2a 0 15min + set-threshold fmcd 0 15min + set-threshold fbcd 0 15min + set-threshold acl-a2n-drop 0 15min + set-threshold acl-n2a-drop 0 15min + set-threshold abra2n 0 1day + set-threshold abrrla2n 0 1day + set-threshold abrn2a 0 1day + set-threshold abrrln2a 0 1day + set-threshold l2cpfd 0 1day + set-threshold bytesina2n 0 1day + set-threshold bytesinn2a 0 1day + set-threshold bytesouta2n 0 1day + set-threshold bytesoutn2a 0 1day + set-threshold fmga2n 0 1day + set-threshold fmgn2a 0 1day + set-threshold fmrda2n 0 1day + set-threshold fmrdn2a 0 1day + set-threshold fmya2n 0 1day + set-threshold fmyn2a 0 1day + set-threshold ftda2n 0 1day + set-threshold ses 0 1day + set-threshold uas 10 1day + set-threshold fdhairpin 0 1day + set-threshold fdstaticblock 0 1day + set-threshold learntblentrydiscards 0 1day + set-threshold numlearntblflushes 0 1day + set-threshold ibra2nmax 0 1day + set-threshold ibrrla2nmax 0 1day + set-threshold ibra2nmin 0 1day + set-threshold ibrrla2nmin 0 1day + set-threshold ibra2n 0 1day + set-threshold ibrrla2n 0 1day + set-threshold ibrn2amax 0 1day + set-threshold ibrrln2amax 0 1day + set-threshold ibrn2amin 0 1day + set-threshold ibrrln2amin 0 1day + set-threshold ibrn2a 0 1day + set-threshold ibrrln2a 0 1day + set-threshold fmcd 0 1day + set-threshold fbcd 0 1day + set-threshold acl-a2n-drop 0 1day + set-threshold acl-n2a-drop 0 1day + exit + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + configure flow flow-1-1-1-3-1 + configure a2n-shaper a2n_shaper-1-1-1-3-1-0 + buffersize 128 + exit + configure a2n-shaper a2n_shaper-1-1-1-3-1-0 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + exit + configure a2n-shaper a2n_shaper-1-1-1-3-1-0 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + configure flow flow-1-1-1-3-1 + configure a2n-policer a2n_policer-1-1-1-3-1-0 + cir 256000 + cbs 1 + eir 194304000 + ebs 3 + color-marking enabled + color-mode color-aware + coupling-flag enabled + exit + configure a2n-policer a2n_policer-1-1-1-3-1-0 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + configure flow flow-1-1-1-3-1 + configure n2a-policer n2a_policer-1-1-1-3-1-0 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + configure n2a-shaper port_n2a_shaper-1-1-1-3-0 + buffersize 1 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + configure flow flow-1-1-1-4-1 + circuit-name "" + es-frame-loss-threshold 1 + ses-frame-loss-threshold-ratio 30 + access-interface access-1-1-1-4 network-interface network-1-1-1-1 push 4-0 none + a2n-shaping-type flow-based + policing enabled + policing-control a2n-n2a + net-to-acc-rate-limiting disabled + default-cos 0 + priority-mapping-profile none + access-learning-ctrl none + network-learning-ctrl none + access-max-forwarding-entries 4096 + network-max-forwarding-entries 4096 + protect-access-learning none + protect-network-learning none + aging-timer 300 + table-full-action forward + n2n-forwarding disabled + a2n-multicast-rate-limit-ctrl disabled + a2n-broadcast-rate-limit-ctrl disabled + a2n-combined-rate-limit-ctrl disabled + eompls-pw none + cpd-filter isl pass-thru + cpd-filter pagp pass-thru + cpd-filter udld pass-thru + cpd-filter cdp pass-thru + cpd-filter vtp pass-thru + cpd-filter dtp pass-thru + cpd-filter pvstp+ pass-thru + cpd-filter uplinkfast pass-thru + cpd-filter vlan-bridge pass-thru + cpd-filter l2pt pass-thru + cpd-filter bpdu pass-thru + cpd-filter pause pass-thru + cpd-filter lacp pass-thru + cpd-filter lacp-marker pass-thru + cpd-filter efm-oam pass-thru + cpd-filter ssm discard + cpd-filter port-authen pass-thru + cpd-filter lan-bridges pass-thru + cpd-filter gmrp pass-thru + cpd-filter gvrp pass-thru + cpd-filter garp pass-thru + cpd-filter elmi pass-thru + cpd-filter 01-80-c2-00-00-00 discard + cpd-filter 01-80-c2-00-00-01 discard + cpd-filter 01-80-c2-00-00-02 discard + cpd-filter 01-80-c2-00-00-03 discard + cpd-filter 01-80-c2-00-00-04 discard + cpd-filter 01-80-c2-00-00-05 discard + cpd-filter 01-80-c2-00-00-06 discard + cpd-filter 01-80-c2-00-00-07 discard + cpd-filter 01-80-c2-00-00-08 discard + cpd-filter 01-80-c2-00-00-09 discard + cpd-filter 01-80-c2-00-00-0a discard + cpd-filter 01-80-c2-00-00-0b discard + cpd-filter 01-80-c2-00-00-0c discard + cpd-filter 01-80-c2-00-00-0d discard + cpd-filter 01-80-c2-00-00-0e discard + cpd-filter 01-80-c2-00-00-0f discard + cpd-filter nearest-lldp discard + cpd-filter non-tpmr-lldp discard + cpd-filter customer-lldp discard + pm + set-threshold abra2n 0 15min + set-threshold abrrla2n 0 15min + set-threshold abrn2a 0 15min + set-threshold abrrln2a 0 15min + set-threshold l2cpfd 0 15min + set-threshold bytesina2n 0 15min + set-threshold bytesinn2a 0 15min + set-threshold bytesouta2n 0 15min + set-threshold bytesoutn2a 0 15min + set-threshold fmga2n 0 15min + set-threshold fmgn2a 0 15min + set-threshold fmrda2n 0 15min + set-threshold fmrdn2a 0 15min + set-threshold fmya2n 0 15min + set-threshold fmyn2a 0 15min + set-threshold ftda2n 0 15min + set-threshold ses 0 15min + set-threshold uas 10 15min + set-threshold fdhairpin 0 15min + set-threshold fdstaticblock 0 15min + set-threshold learntblentrydiscards 0 15min + set-threshold numlearntblflushes 0 15min + set-threshold ibra2nmax 0 15min + set-threshold ibrrla2nmax 0 15min + set-threshold ibra2nmin 0 15min + set-threshold ibrrla2nmin 0 15min + set-threshold ibra2n 0 15min + set-threshold ibrrla2n 0 15min + set-threshold ibrn2amax 0 15min + set-threshold ibrrln2amax 0 15min + set-threshold ibrn2amin 0 15min + set-threshold ibrrln2amin 0 15min + set-threshold ibrn2a 0 15min + set-threshold ibrrln2a 0 15min + set-threshold fmcd 0 15min + set-threshold fbcd 0 15min + set-threshold acl-a2n-drop 0 15min + set-threshold acl-n2a-drop 0 15min + set-threshold abra2n 0 1day + set-threshold abrrla2n 0 1day + set-threshold abrn2a 0 1day + set-threshold abrrln2a 0 1day + set-threshold l2cpfd 0 1day + set-threshold bytesina2n 0 1day + set-threshold bytesinn2a 0 1day + set-threshold bytesouta2n 0 1day + set-threshold bytesoutn2a 0 1day + set-threshold fmga2n 0 1day + set-threshold fmgn2a 0 1day + set-threshold fmrda2n 0 1day + set-threshold fmrdn2a 0 1day + set-threshold fmya2n 0 1day + set-threshold fmyn2a 0 1day + set-threshold ftda2n 0 1day + set-threshold ses 0 1day + set-threshold uas 10 1day + set-threshold fdhairpin 0 1day + set-threshold fdstaticblock 0 1day + set-threshold learntblentrydiscards 0 1day + set-threshold numlearntblflushes 0 1day + set-threshold ibra2nmax 0 1day + set-threshold ibrrla2nmax 0 1day + set-threshold ibra2nmin 0 1day + set-threshold ibrrla2nmin 0 1day + set-threshold ibra2n 0 1day + set-threshold ibrrla2n 0 1day + set-threshold ibrn2amax 0 1day + set-threshold ibrrln2amax 0 1day + set-threshold ibrn2amin 0 1day + set-threshold ibrrln2amin 0 1day + set-threshold ibrn2a 0 1day + set-threshold ibrrln2a 0 1day + set-threshold fmcd 0 1day + set-threshold fbcd 0 1day + set-threshold acl-a2n-drop 0 1day + set-threshold acl-n2a-drop 0 1day + exit + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + configure flow flow-1-1-1-4-1 + configure a2n-shaper a2n_shaper-1-1-1-4-1-0 + buffersize 128 + exit + configure a2n-shaper a2n_shaper-1-1-1-4-1-0 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + exit + configure a2n-shaper a2n_shaper-1-1-1-4-1-0 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + configure flow flow-1-1-1-4-1 + configure a2n-policer a2n_policer-1-1-1-4-1-0 + eir 0 + ebs 0 + cir 0 + cbs 0 + color-marking enabled + color-mode color-aware + coupling-flag enabled + exit + configure a2n-policer a2n_policer-1-1-1-4-1-0 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + configure flow flow-1-1-1-4-1 + configure n2a-policer n2a_policer-1-1-1-4-1-0 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + configure n2a-shaper port_n2a_shaper-1-1-1-4-0 + buffersize 0 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + configure flow flow-1-1-1-5-1 + circuit-name "" + es-frame-loss-threshold 1 + ses-frame-loss-threshold-ratio 30 + access-interface access-1-1-1-5 network-interface network-1-1-1-1 push 5-0 none + a2n-shaping-type flow-based + policing enabled + policing-control a2n-n2a + net-to-acc-rate-limiting disabled + default-cos 0 + priority-mapping-profile none + access-learning-ctrl none + network-learning-ctrl none + access-max-forwarding-entries 4096 + network-max-forwarding-entries 4096 + protect-access-learning none + protect-network-learning none + aging-timer 300 + table-full-action forward + n2n-forwarding disabled + a2n-multicast-rate-limit-ctrl disabled + a2n-broadcast-rate-limit-ctrl disabled + a2n-combined-rate-limit-ctrl disabled + eompls-pw none + cpd-filter isl pass-thru + cpd-filter pagp pass-thru + cpd-filter udld pass-thru + cpd-filter cdp pass-thru + cpd-filter vtp pass-thru + cpd-filter dtp pass-thru + cpd-filter pvstp+ pass-thru + cpd-filter uplinkfast pass-thru + cpd-filter vlan-bridge pass-thru + cpd-filter l2pt pass-thru + cpd-filter bpdu pass-thru + cpd-filter pause pass-thru + cpd-filter lacp pass-thru + cpd-filter lacp-marker pass-thru + cpd-filter efm-oam pass-thru + cpd-filter ssm discard + cpd-filter port-authen pass-thru + cpd-filter lan-bridges pass-thru + cpd-filter gmrp pass-thru + cpd-filter gvrp pass-thru + cpd-filter garp pass-thru + cpd-filter elmi pass-thru + cpd-filter 01-80-c2-00-00-00 discard + cpd-filter 01-80-c2-00-00-01 discard + cpd-filter 01-80-c2-00-00-02 discard + cpd-filter 01-80-c2-00-00-03 discard + cpd-filter 01-80-c2-00-00-04 discard + cpd-filter 01-80-c2-00-00-05 discard + cpd-filter 01-80-c2-00-00-06 discard + cpd-filter 01-80-c2-00-00-07 discard + cpd-filter 01-80-c2-00-00-08 discard + cpd-filter 01-80-c2-00-00-09 discard + cpd-filter 01-80-c2-00-00-0a discard + cpd-filter 01-80-c2-00-00-0b discard + cpd-filter 01-80-c2-00-00-0c discard + cpd-filter 01-80-c2-00-00-0d discard + cpd-filter 01-80-c2-00-00-0e discard + cpd-filter 01-80-c2-00-00-0f discard + cpd-filter nearest-lldp discard + cpd-filter non-tpmr-lldp discard + cpd-filter customer-lldp discard + pm + set-threshold abra2n 0 15min + set-threshold abrrla2n 0 15min + set-threshold abrn2a 0 15min + set-threshold abrrln2a 0 15min + set-threshold l2cpfd 0 15min + set-threshold bytesina2n 0 15min + set-threshold bytesinn2a 0 15min + set-threshold bytesouta2n 0 15min + set-threshold bytesoutn2a 0 15min + set-threshold fmga2n 0 15min + set-threshold fmgn2a 0 15min + set-threshold fmrda2n 0 15min + set-threshold fmrdn2a 0 15min + set-threshold fmya2n 0 15min + set-threshold fmyn2a 0 15min + set-threshold ftda2n 0 15min + set-threshold ses 0 15min + set-threshold uas 10 15min + set-threshold fdhairpin 0 15min + set-threshold fdstaticblock 0 15min + set-threshold learntblentrydiscards 0 15min + set-threshold numlearntblflushes 0 15min + set-threshold ibra2nmax 0 15min + set-threshold ibrrla2nmax 0 15min + set-threshold ibra2nmin 0 15min + set-threshold ibrrla2nmin 0 15min + set-threshold ibra2n 0 15min + set-threshold ibrrla2n 0 15min + set-threshold ibrn2amax 0 15min + set-threshold ibrrln2amax 0 15min + set-threshold ibrn2amin 0 15min + set-threshold ibrrln2amin 0 15min + set-threshold ibrn2a 0 15min + set-threshold ibrrln2a 0 15min + set-threshold fmcd 0 15min + set-threshold fbcd 0 15min + set-threshold acl-a2n-drop 0 15min + set-threshold acl-n2a-drop 0 15min + set-threshold abra2n 0 1day + set-threshold abrrla2n 0 1day + set-threshold abrn2a 0 1day + set-threshold abrrln2a 0 1day + set-threshold l2cpfd 0 1day + set-threshold bytesina2n 0 1day + set-threshold bytesinn2a 0 1day + set-threshold bytesouta2n 0 1day + set-threshold bytesoutn2a 0 1day + set-threshold fmga2n 0 1day + set-threshold fmgn2a 0 1day + set-threshold fmrda2n 0 1day + set-threshold fmrdn2a 0 1day + set-threshold fmya2n 0 1day + set-threshold fmyn2a 0 1day + set-threshold ftda2n 0 1day + set-threshold ses 0 1day + set-threshold uas 10 1day + set-threshold fdhairpin 0 1day + set-threshold fdstaticblock 0 1day + set-threshold learntblentrydiscards 0 1day + set-threshold numlearntblflushes 0 1day + set-threshold ibra2nmax 0 1day + set-threshold ibrrla2nmax 0 1day + set-threshold ibra2nmin 0 1day + set-threshold ibrrla2nmin 0 1day + set-threshold ibra2n 0 1day + set-threshold ibrrla2n 0 1day + set-threshold ibrn2amax 0 1day + set-threshold ibrrln2amax 0 1day + set-threshold ibrn2amin 0 1day + set-threshold ibrrln2amin 0 1day + set-threshold ibrn2a 0 1day + set-threshold ibrrln2a 0 1day + set-threshold fmcd 0 1day + set-threshold fbcd 0 1day + set-threshold acl-a2n-drop 0 1day + set-threshold acl-n2a-drop 0 1day + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + configure flow flow-1-1-1-5-1 + configure a2n-shaper a2n_shaper-1-1-1-5-1-0 + buffersize 128 + exit + configure a2n-shaper a2n_shaper-1-1-1-5-1-0 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + exit + configure a2n-shaper a2n_shaper-1-1-1-5-1-0 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + configure flow flow-1-1-1-5-1 + configure a2n-policer a2n_policer-1-1-1-5-1-0 + eir 0 + ebs 0 + cir 0 + cbs 0 + color-marking enabled + color-mode color-aware + coupling-flag enabled + exit + configure a2n-policer a2n_policer-1-1-1-5-1-0 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + configure flow flow-1-1-1-5-1 + configure n2a-policer n2a_policer-1-1-1-5-1-0 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + configure n2a-shaper port_n2a_shaper-1-1-1-5-0 + buffersize 0 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + add flow flow-1-1-1-6-1 "Y.1731 Test Port" regular-evc disabled disabled disabled enabled 0 outer-vlantag enabled none none "506-*" 0 64000 eompls-pw none prio_map_profile-1 access-interface access-1-1-1-6 network-interface network-1-1-1-1 flow-based + configure flow flow-1-1-1-6-1 + admin-state management + hierarchical-cos disabled + es-frame-loss-threshold 1 + ses-frame-loss-threshold-ratio 30 + policing enabled + policing-control a2n-n2a + n2a-outertag-prio-ctrl disabled + access-learning-ctrl none + network-learning-ctrl none + access-max-forwarding-entries 4096 + network-max-forwarding-entries 4096 + protect-access-learning none + protect-network-learning none + aging-timer 300 + table-full-action forward + n2n-forwarding disabled + a2n-multicast-rate-limit-ctrl disabled + a2n-broadcast-rate-limit-ctrl disabled + a2n-combined-rate-limit-ctrl disabled + cpd-filter isl pass-thru + cpd-filter pagp pass-thru + cpd-filter udld pass-thru + cpd-filter cdp pass-thru + cpd-filter vtp pass-thru + cpd-filter dtp pass-thru + cpd-filter pvstp+ pass-thru + cpd-filter uplinkfast pass-thru + cpd-filter vlan-bridge pass-thru + cpd-filter l2pt pass-thru + cpd-filter bpdu pass-thru + cpd-filter pause pass-thru + cpd-filter lacp pass-thru + cpd-filter lacp-marker pass-thru + cpd-filter efm-oam pass-thru + cpd-filter ssm discard + cpd-filter port-authen pass-thru + cpd-filter lan-bridges pass-thru + cpd-filter gmrp pass-thru + cpd-filter gvrp pass-thru + cpd-filter garp pass-thru + cpd-filter elmi pass-thru + cpd-filter 01-80-c2-00-00-00 discard + cpd-filter 01-80-c2-00-00-01 discard + cpd-filter 01-80-c2-00-00-02 discard + cpd-filter 01-80-c2-00-00-03 discard + cpd-filter 01-80-c2-00-00-04 discard + cpd-filter 01-80-c2-00-00-05 discard + cpd-filter 01-80-c2-00-00-06 discard + cpd-filter 01-80-c2-00-00-07 discard + cpd-filter 01-80-c2-00-00-08 discard + cpd-filter 01-80-c2-00-00-09 discard + cpd-filter 01-80-c2-00-00-0a discard + cpd-filter 01-80-c2-00-00-0b discard + cpd-filter 01-80-c2-00-00-0c discard + cpd-filter 01-80-c2-00-00-0d discard + cpd-filter 01-80-c2-00-00-0e discard + cpd-filter 01-80-c2-00-00-0f discard + cpd-filter nearest-lldp discard + cpd-filter non-tpmr-lldp discard + cpd-filter customer-lldp discard + pm + set-threshold abra2n 0 15min + set-threshold abrrla2n 0 15min + set-threshold abrn2a 0 15min + set-threshold abrrln2a 0 15min + set-threshold l2cpfd 0 15min + set-threshold bytesina2n 0 15min + set-threshold bytesinn2a 0 15min + set-threshold bytesouta2n 0 15min + set-threshold bytesoutn2a 0 15min + set-threshold fmga2n 0 15min + set-threshold fmgn2a 0 15min + set-threshold fmrda2n 0 15min + set-threshold fmrdn2a 0 15min + set-threshold fmya2n 0 15min + set-threshold fmyn2a 0 15min + set-threshold ftda2n 0 15min + set-threshold ses 0 15min + set-threshold uas 10 15min + set-threshold fdhairpin 0 15min + set-threshold fdstaticblock 0 15min + set-threshold learntblentrydiscards 0 15min + set-threshold numlearntblflushes 0 15min + set-threshold ibra2nmax 0 15min + set-threshold ibrrla2nmax 0 15min + set-threshold ibra2nmin 0 15min + set-threshold ibrrla2nmin 0 15min + set-threshold ibra2n 0 15min + set-threshold ibrrla2n 0 15min + set-threshold ibrn2amax 0 15min + set-threshold ibrrln2amax 0 15min + set-threshold ibrn2amin 0 15min + set-threshold ibrrln2amin 0 15min + set-threshold ibrn2a 0 15min + set-threshold ibrrln2a 0 15min + set-threshold fmcd 0 15min + set-threshold fbcd 0 15min + set-threshold acl-a2n-drop 0 15min + set-threshold acl-n2a-drop 0 15min + set-threshold abra2n 0 1day + set-threshold abrrla2n 0 1day + set-threshold abrn2a 0 1day + set-threshold abrrln2a 0 1day + set-threshold l2cpfd 0 1day + set-threshold bytesina2n 0 1day + set-threshold bytesinn2a 0 1day + set-threshold bytesouta2n 0 1day + set-threshold bytesoutn2a 0 1day + set-threshold fmga2n 0 1day + set-threshold fmgn2a 0 1day + set-threshold fmrda2n 0 1day + set-threshold fmrdn2a 0 1day + set-threshold fmya2n 0 1day + set-threshold fmyn2a 0 1day + set-threshold ftda2n 0 1day + set-threshold ses 0 1day + set-threshold uas 10 1day + set-threshold fdhairpin 0 1day + set-threshold fdstaticblock 0 1day + set-threshold learntblentrydiscards 0 1day + set-threshold numlearntblflushes 0 1day + set-threshold ibra2nmax 0 1day + set-threshold ibrrla2nmax 0 1day + set-threshold ibra2nmin 0 1day + set-threshold ibrrla2nmin 0 1day + set-threshold ibra2n 0 1day + set-threshold ibrrla2n 0 1day + set-threshold ibrn2amax 0 1day + set-threshold ibrrln2amax 0 1day + set-threshold ibrn2amin 0 1day + set-threshold ibrrln2amin 0 1day + set-threshold ibrn2a 0 1day + set-threshold ibrrln2a 0 1day + set-threshold fmcd 0 1day + set-threshold fbcd 0 1day + set-threshold acl-a2n-drop 0 1day + set-threshold acl-n2a-drop 0 1day + exit + exit + exit + exit + exit + + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + configure a2n-shaper a2n_shaper-1-1-1-6-1-0 + buffersize 128 + exit + configure a2n-shaper a2n_shaper-1-1-1-6-1-0 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + exit + configure a2n-shaper a2n_shaper-1-1-1-6-1-0 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + exit + + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-shaper a2n_shaper-1-1-1-6-1-1 128 + configure a2n-shaper a2n_shaper-1-1-1-6-1-1 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + exit + configure a2n-shaper a2n_shaper-1-1-1-6-1-1 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-shaper a2n_shaper-1-1-1-6-1-2 128 + configure a2n-shaper a2n_shaper-1-1-1-6-1-2 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + exit + configure a2n-shaper a2n_shaper-1-1-1-6-1-2 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-shaper a2n_shaper-1-1-1-6-1-3 128 + configure a2n-shaper a2n_shaper-1-1-1-6-1-3 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + exit + configure a2n-shaper a2n_shaper-1-1-1-6-1-3 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-shaper a2n_shaper-1-1-1-6-1-4 128 + configure a2n-shaper a2n_shaper-1-1-1-6-1-4 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + exit + configure a2n-shaper a2n_shaper-1-1-1-6-1-4 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-shaper a2n_shaper-1-1-1-6-1-5 128 + configure a2n-shaper a2n_shaper-1-1-1-6-1-5 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + exit + configure a2n-shaper a2n_shaper-1-1-1-6-1-5 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-shaper a2n_shaper-1-1-1-6-1-6 128 + configure a2n-shaper a2n_shaper-1-1-1-6-1-6 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + exit + configure a2n-shaper a2n_shaper-1-1-1-6-1-6 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-shaper a2n_shaper-1-1-1-6-1-7 128 + configure a2n-shaper a2n_shaper-1-1-1-6-1-7 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + exit + configure a2n-shaper a2n_shaper-1-1-1-6-1-7 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + configure a2n-policer a2n_policer-1-1-1-6-1-0 + eir 64000 + ebs 32 + cir 0 + cbs 0 + color-marking enabled + color-mode color-blind + coupling-flag enabled + exit + configure a2n-policer a2n_policer-1-1-1-6-1-0 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-policer a2n_policer-1-1-1-6-1-1 enabled color-blind enabled 0 64000 0 32 a2n_shaper-1-1-1-6-1-1 + configure a2n-policer a2n_policer-1-1-1-6-1-1 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-policer a2n_policer-1-1-1-6-1-2 enabled color-blind enabled 0 64000 0 32 a2n_shaper-1-1-1-6-1-2 + configure a2n-policer a2n_policer-1-1-1-6-1-2 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-policer a2n_policer-1-1-1-6-1-3 enabled color-blind enabled 0 64000 0 32 a2n_shaper-1-1-1-6-1-3 + configure a2n-policer a2n_policer-1-1-1-6-1-3 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-policer a2n_policer-1-1-1-6-1-4 enabled color-blind enabled 0 64000 0 32 a2n_shaper-1-1-1-6-1-4 + configure a2n-policer a2n_policer-1-1-1-6-1-4 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-policer a2n_policer-1-1-1-6-1-5 enabled color-blind enabled 0 64000 0 32 a2n_shaper-1-1-1-6-1-5 + configure a2n-policer a2n_policer-1-1-1-6-1-5 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-policer a2n_policer-1-1-1-6-1-6 enabled color-blind enabled 0 64000 0 32 a2n_shaper-1-1-1-6-1-6 + configure a2n-policer a2n_policer-1-1-1-6-1-6 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add a2n-policer a2n_policer-1-1-1-6-1-7 enabled color-blind enabled 0 64000 0 32 a2n_shaper-1-1-1-6-1-7 + configure a2n-policer a2n_policer-1-1-1-6-1-7 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure n2a-shaper port_n2a_shaper-1-1-1-6-0 + buffersize 10 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + configure n2a-policer n2a_policer-1-1-1-6-1-0 + eir 64000 + ebs 32 + cir 0 + cbs 0 + color-marking enabled + color-mode color-blind + coupling-flag enabled + exit + configure n2a-policer n2a_policer-1-1-1-6-1-0 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add n2a-policer n2a_policer-1-1-1-6-1-1 enabled color-blind enabled 0 64000 0 32 + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure n2a-shaper port_n2a_shaper-1-1-1-6-1 + buffersize 10 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + configure n2a-policer n2a_policer-1-1-1-6-1-1 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add n2a-policer n2a_policer-1-1-1-6-1-2 enabled color-blind enabled 0 64000 0 32 + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure n2a-shaper port_n2a_shaper-1-1-1-6-2 + buffersize 10 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + configure n2a-policer n2a_policer-1-1-1-6-1-2 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add n2a-policer n2a_policer-1-1-1-6-1-3 enabled color-blind enabled 0 64000 0 32 + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure n2a-shaper port_n2a_shaper-1-1-1-6-3 + buffersize 10 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + configure n2a-policer n2a_policer-1-1-1-6-1-3 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add n2a-policer n2a_policer-1-1-1-6-1-4 enabled color-blind enabled 0 64000 0 32 + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure n2a-shaper port_n2a_shaper-1-1-1-6-4 + buffersize 10 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + configure n2a-policer n2a_policer-1-1-1-6-1-4 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add n2a-policer n2a_policer-1-1-1-6-1-5 enabled color-blind enabled 0 64000 0 32 + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure n2a-shaper port_n2a_shaper-1-1-1-6-5 + buffersize 10 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + configure n2a-policer n2a_policer-1-1-1-6-1-5 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add n2a-policer n2a_policer-1-1-1-6-1-6 enabled color-blind enabled 0 64000 0 32 + exit + exit + exit + exit + + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure n2a-shaper port_n2a_shaper-1-1-1-6-6 + buffersize 10 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + configure n2a-policer n2a_policer-1-1-1-6-1-6 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + add n2a-policer n2a_policer-1-1-1-6-1-7 enabled color-blind enabled 0 64000 0 32 + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure n2a-shaper port_n2a_shaper-1-1-1-6-7 + buffersize 10 + soam-cir 0 + soam-eir 0 + max-drop-probability-green 100 + max-threshold-green 100 + min-threshold-green 100 + max-drop-probability-yellow 100 + max-threshold-yellow 75 + min-threshold-yellow 75 + pm + set-threshold bt 0 15min + set-threshold btd 0 15min + set-threshold fd 0 15min + set-threshold ftd 0 15min + set-threshold abrrl 0 15min + set-threshold bredd 0 15min + set-threshold fredd 0 15min + set-threshold bt 0 1day + set-threshold btd 0 1day + set-threshold fd 0 1day + set-threshold ftd 0 1day + set-threshold abrrl 0 1day + set-threshold bredd 0 1day + set-threshold fredd 0 1day + exit + exit + exit + exit + exit + + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + configure flow flow-1-1-1-6-1 + configure n2a-policer n2a_policer-1-1-1-6-1-7 + pm + set-threshold fmg 0 15min + set-threshold fmy 0 15min + set-threshold fmrd 0 15min + set-threshold abr 0 15min + set-threshold bytesin 0 15min + set-threshold bytesout 0 15min + set-threshold fmg 0 1day + set-threshold fmy 0 1day + set-threshold fmrd 0 1day + set-threshold abr 0 1day + set-threshold bytesin 0 1day + set-threshold bytesout 0 1day + exit + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-1 + lpbk + rls-lpbk + exit + exit + exit + exit + + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-2 + lpbk + rls-lpbk + exit + exit + exit + exit + + + adva-825:configure cfm + signal-fail-defect-list rdi,rem-ccm + exit + + + adva-825:configure cfm + configure sysdef-md + md-level 0 + mhf none + exit + exit + + + adva-825:configure cfm + add md 1 no-name 5 none + exit + + + adva-825:configure cfm + configure md md-1 + add manet 1 icc-based "CBLM_METRO" 1sec 1 + exit + exit + + + adva-825:configure cfm + configure md md-1 + configure manet manet-1-1 + add macomp 1 access-1-1-1-6 506 defer + add mep 1 up access-1-1-1-6 in-service mac-remote-xcon-error disabled 0 + configure mep mep-1-1-1 + ais-client-mdlevel 6 + ais-gen disabled + ais-interval 1sec + ais-prio 0 + ais-trigger none + ethernet-type 0x8100 + llf-trigger none + lm-dualended-countallprios disabled + lm-rx-countallprios disabled + lm-tx-countallprios disabled + lm-in-profile enabled + primary-vid 0 + sat-responder disabled + ltm-egress-id "00000080ea72eaf0" + exit + exit + exit + exit + + + adva-825:configure cfm + configure cfm-n2a-vid-shaper access-1-1-1-6 cfm_n2a_vid_shaper-1-1-1-6-1 + buffersize 1 + cir 320000 + admin-state management + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure usb + add mobile-modem + exit + configure usb + configure mobile-modem + admin-state unassigned + alias "" + apn "" + dial-number "" + redial-timer 10 + exit + exit + exit + exit + + + adva-825:admin sw-license + configure feature "OpenFlow" disabled + exit + + + adva-825:admin sw-license + configure feature "IP-Services-and-ACL-Filtering" disabled + exit + + adva-825:admin sw-license + configure feature "Extended-Scale-IP-services" disabled + exit + + + adva-825:admin sw-license + configure feature "EoMPLS" disabled + exit + + adva-825:configure system + lldp + trans-interval 30 + holdtime-multiplier 4 + reinit-delay 2 + notification-interval 30 + tx-credit-max 5 + message-fast-tx 1 + tx-fast-init 4 + max-neighbor-action discard-new + exit + exit + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-3 + lldp + add acc-port-config 1 + exit + lldp + configure acc-port-config 1 + admin-status disabled + snmp-notification disabled + basic-tlv-supported port-description,sys-name,sys-description,sys-cap + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-4 + lldp + add acc-port-config 1 + exit + lldp + configure acc-port-config 1 + admin-status disabled + snmp-notification disabled + basic-tlv-supported port-description,sys-name,sys-description,sys-cap + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-5 + lldp + add acc-port-config 1 + exit + lldp + configure acc-port-config 1 + admin-status disabled + snmp-notification disabled + basic-tlv-supported port-description,sys-name,sys-description,sys-cap + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure access-port access-1-1-1-6 + lldp + add acc-port-config 1 + exit + lldp + configure acc-port-config 1 + admin-status disabled + snmp-notification disabled + basic-tlv-supported port-description,sys-name,sys-description,sys-cap + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-1 + lldp + add net-port-config 1 + exit + lldp + configure net-port-config 1 + admin-status disabled + snmp-notification disabled + basic-tlv-supported port-description,sys-name,sys-description,sys-cap + exit + exit + exit + exit + exit + + + adva-825:network-element ne-1 + configure nte nte114pro_he-1-1-1 + configure network-port network-1-1-1-2 + lldp + add net-port-config 1 + exit + lldp + configure net-port-config 1 + admin-status disabled + snmp-notification disabled + basic-tlv-supported port-description,sys-name,sys-description,sys-cap + exit + exit + exit + exit + exit + + adva-825:configure sat + network-element ne-1 + configure sat-ctrl sat_control-1-1 + name "" + test-mode one-way + test-procedure cir,eir,policing,performance + test-duration 20 + step-duration 20 + step-num 1 + perf-test-duration 15 + flpdu-tag-override disabled + exit + exit + exit + + + adva-825:configure communication + add ip-route nexthop 0.0.0.0 0.0.0.0 1.1.1.1 "eth0" 1 disabled + exit + + adva-825:configure system + profile + add prio-map-profile 2 "IPVPN Managed and Unmanaged Premium 5 Class" ip-dscp + exit + profile + configure prio-map-profile prio_map_profile-2 + cos-map-mode ethernet + edit-prio-map prio_map_profile_pri_entry-2-1 0 none + edit-prio-map prio_map_profile_pri_entry-2-2 0 none + edit-prio-map prio_map_profile_pri_entry-2-3 0 none + edit-prio-map prio_map_profile_pri_entry-2-4 0 none + edit-prio-map prio_map_profile_pri_entry-2-5 0 none + edit-prio-map prio_map_profile_pri_entry-2-6 0 none + edit-prio-map prio_map_profile_pri_entry-2-7 0 none + edit-prio-map prio_map_profile_pri_entry-2-8 0 none + edit-prio-map prio_map_profile_pri_entry-2-9 1 none + edit-prio-map prio_map_profile_pri_entry-2-10 1 none + edit-prio-map prio_map_profile_pri_entry-2-11 1 none + edit-prio-map prio_map_profile_pri_entry-2-12 1 none + edit-prio-map prio_map_profile_pri_entry-2-13 1 none + edit-prio-map prio_map_profile_pri_entry-2-14 1 none + edit-prio-map prio_map_profile_pri_entry-2-15 1 none + edit-prio-map prio_map_profile_pri_entry-2-16 1 none + edit-prio-map prio_map_profile_pri_entry-2-17 2 none + edit-prio-map prio_map_profile_pri_entry-2-18 2 none + edit-prio-map prio_map_profile_pri_entry-2-19 2 none + edit-prio-map prio_map_profile_pri_entry-2-20 2 none + edit-prio-map prio_map_profile_pri_entry-2-21 2 none + edit-prio-map prio_map_profile_pri_entry-2-22 2 none + edit-prio-map prio_map_profile_pri_entry-2-23 2 none + edit-prio-map prio_map_profile_pri_entry-2-24 2 none + edit-prio-map prio_map_profile_pri_entry-2-25 3 none + edit-prio-map prio_map_profile_pri_entry-2-26 3 none + edit-prio-map prio_map_profile_pri_entry-2-27 3 none + edit-prio-map prio_map_profile_pri_entry-2-28 3 none + edit-prio-map prio_map_profile_pri_entry-2-29 3 none + edit-prio-map prio_map_profile_pri_entry-2-30 3 none + edit-prio-map prio_map_profile_pri_entry-2-31 3 none + edit-prio-map prio_map_profile_pri_entry-2-32 3 none + edit-prio-map prio_map_profile_pri_entry-2-33 4 none + edit-prio-map prio_map_profile_pri_entry-2-34 4 none + edit-prio-map prio_map_profile_pri_entry-2-35 4 none + edit-prio-map prio_map_profile_pri_entry-2-36 4 none + edit-prio-map prio_map_profile_pri_entry-2-37 4 none + edit-prio-map prio_map_profile_pri_entry-2-38 4 none + edit-prio-map prio_map_profile_pri_entry-2-39 4 none + edit-prio-map prio_map_profile_pri_entry-2-40 4 none + edit-prio-map prio_map_profile_pri_entry-2-41 5 none + edit-prio-map prio_map_profile_pri_entry-2-42 5 none + edit-prio-map prio_map_profile_pri_entry-2-43 5 none + edit-prio-map prio_map_profile_pri_entry-2-44 5 none + edit-prio-map prio_map_profile_pri_entry-2-45 5 none + edit-prio-map prio_map_profile_pri_entry-2-46 5 none + edit-prio-map prio_map_profile_pri_entry-2-47 5 none + edit-prio-map prio_map_profile_pri_entry-2-48 5 none + edit-prio-map prio_map_profile_pri_entry-2-49 6 none + edit-prio-map prio_map_profile_pri_entry-2-50 6 none + edit-prio-map prio_map_profile_pri_entry-2-51 6 none + edit-prio-map prio_map_profile_pri_entry-2-52 6 none + edit-prio-map prio_map_profile_pri_entry-2-53 6 none + edit-prio-map prio_map_profile_pri_entry-2-54 6 none + edit-prio-map prio_map_profile_pri_entry-2-55 6 none + edit-prio-map prio_map_profile_pri_entry-2-56 6 none + edit-prio-map prio_map_profile_pri_entry-2-57 7 none + edit-prio-map prio_map_profile_pri_entry-2-58 7 none + edit-prio-map prio_map_profile_pri_entry-2-59 7 none + edit-prio-map prio_map_profile_pri_entry-2-60 7 none + edit-prio-map prio_map_profile_pri_entry-2-61 7 none + edit-prio-map prio_map_profile_pri_entry-2-62 7 none + edit-prio-map prio_map_profile_pri_entry-2-63 7 none + edit-prio-map prio_map_profile_pri_entry-2-64 7 none + edit-cos-map 0 0 0 + edit-cos-map 1 1 1 + edit-cos-map 2 2 2 + edit-cos-map 3 3 3 + edit-cos-map 4 4 4 + edit-cos-map 5 5 5 + edit-cos-map 6 6 6 + edit-cos-map 7 7 7 + exit + exit + exit + ``` + + +# 5. Built in live-status actions +--------------------------------- + + The NED supports the following live-status actions: + ``` + admin@ncs(config-device-adva-1)# live-status exec show + ``` + + +# 6. Built in live-status show +------------------------------ + + The NED does not implement any TTL'based live-status data + + +# 7. Limitations +---------------- + + NONE + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings adva-825 logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings adva-825 logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` diff --git a/alu-isam/README-ned-settings.md b/alu-isam/README-ned-settings.md new file mode 100644 index 00000000..f1d7f042 --- /dev/null +++ b/alu-isam/README-ned-settings.md @@ -0,0 +1,471 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/alu-isam/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/alu-isam/ + device + /ncs:/device/devices/device:/ned-settings/alu-isam/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings alu-isam + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings alu-isam + 2. developer-settings + 3. developer + 4. deprecated + 5. write + 5.1. config-dependency + 6. read + 6.1. replace-config + 7. alu-isam-config-warning + 8. alu-isam-extra-errors + 9. connection + 10. logger + 11. proxy + ``` + + +# 1. ned-settings alu-isam +-------------------------- + + + - alu-isam extended-parser (default auto) + + Make the alu-isam NED enable extensions to ease the task of the NCS/NSO CLI command parser. A + common problem with the alu-isam NED is that can get out of sync when trying to parse config + that is not supported by the YANG model. This is especially the case for unsupported config + that generates a mode switch. + + disabled - Load configuration the standard way. + + turbo-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO using + maapi setvalues call. + + turbo-xml-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO in XML + format. + + robust-mode - Makes the NED filter the configuration so that unmodeled content is removed + before being passed to the NSO CLI-engine. This protects against + configuration ending up at the wrong level when NSO CLI parser fallbacks + (which potentially can cause following config to be skipped). + + old-robust-mode - Makes the NED alter the config dump such that all mode switches are always + done from top and down instead of from below and up (with the 'exit' + command) before given to the NCS/NSO parser. +The number of lines in the + config dump will increase a lot with this feature enabled. + + auto - Uses turbo-mode when available, will use fastest availablemethod to load + data to NSO. If NSO doesn't support data-loading from CLI NED, robust-mode + is used. (default). + + + - alu-isam trans-id-method (default set-log-entry-sequence-number) + + Configure how the NED shall calculate the transaction id. Typically used after each commit and + for check-sync operations. + + config-hash - Use a snapshot of the running config for + calculation.(default). + + set-log-entry-sequence-number - Use the sequence number of the latest entry in the ISAM + transaction set log for calculation. + + + - alu-isam disable-interface-admin-state (default false) + + This ned-setting can be used to disable all equipment/ont/interface/admin-state yang nodes. + You can use this ned-settings, to skip admin-state during commit and compare-config/sync-from + + # devices global-settings ned-settings alu-isam disable-interface-admin-state true + or + # devices device isamdev ned-settings alu-isam disable-interface-admin-state true + # commit + # disconnect + # sync-from + + Note: in order for the above settings to take effect, user must disconnect and do sync-from. + + +# 2. ned-settings alu-isam developer-settings +--------------------------------------------- + + Contains settings used by the NED developers. + + + - developer-settings load-from-file + + Make the NED load a file containing raw device config when doing sync-from. Does only work on + NETSIM targets. + + +# 3. ned-settings alu-isam developer +------------------------------------ + + Contains settings used for debugging (intended for NED developers). + + + - developer progress-verbosity (default disabled) + + Maximum NED verbosity level which will be reported. + + disabled - disabled. + + normal - normal. + + verbose - verbose. + + very-verbose - very-verbose. + + debug - debug. + + + - developer platform model + + Override device model name/number. + + + - developer platform name + + Override device name. + + + - developer platform version + + Override device version. + + +# 4. ned-settings alu-isam deprecated +------------------------------------- + + Deprecated ned-settings. + + + - deprecated connection legacy-mode (default enabled) + + enabled - enabled. + + disabled - disabled. + + +# 5. ned-settings alu-isam write +-------------------------------- + + Settings used when writing to device. + + +## 5.1. ned-settings alu-isam write config-dependency +----------------------------------------------------- + + This ned-setting can be used to add dynamic dependency rules to + the NED before being permanently fixed in the NED. This can be + useful if a dependency bug is found and you do not want to upgrade + the NED or are in a hurry for the fix. + + - write config-dependency + + - id + + List id, any string. + + - mode + + Regex specifying config mode where the rule is checked, don't set for top-mode. + + - move + + Regex|match-expr specifying line(s) to move. + + - action + + before - Move 'move' line(s) before 'stay' line(s). + + after - Move 'move' line(s) before 'stay' line(s). + + last - Move 'move' line(s) last. + + first - Move 'move' line(s) first. + + - stay + + Regex|match-expr specifying where 'move' lines will be moved before|after. + + - options + + Optional rule option(s). + + Note that in order for the this ned-setting to take effect, you + must disconnect and connect again. + + +# 6. ned-settings alu-isam read +------------------------------- + + Settings used when reading from device. + + +## 6.1. ned-settings alu-isam read replace-config +------------------------------------------------- + + The replace-config list ned-setting can be used to + replace or filter out config line(s) upon reading from device + + - read replace-config + + - id + + List id, any string. + + - regexp + + The regular expression (DOTALL) to which the config is to be matched. + + - replacement + + The string which would replace all found matches. May use groups from regex. + + An example, on one device the config line "sernum ALCL:F961418E" + which is a auto-generated config from device. This has to be filtered out. + This can be done with the following ned-setting: + #devices device ned-settings alu-isam read replace-config sernum regexp "\\n\\s+sernum ALCL:F961418E" + The above ned-setting will result in the line being stripped if available. + Note how the replacement string is left empty when filtering. This + means replacing with "". + The NED trace (in raw mode) will show the ned-setting in use when + doing a check-sync or sync-from: + -- transformed <= replaced "\n sernum ALCL:F961418E" with "" + Finally, a word of warning, if you replace or filter out config + from the show running-config, you most likely will have difficulties modifying this config + Note: in order for the above settings to take effect, user must disconnect and do connect. + + +# 7. ned-settings alu-isam alu-isam-config-warning +-------------------------------------------------- + + This section describes how device output (warnings/errors) can be filtered. + + - alu-isam alu-isam-config-warning + + - warning + + Warning regular expression, e.g. .* does not exist.*. + + After having sent a config command to the device the NED will treat + any text reply as an error and abort the transaction. The config + command that caused the failed transaction will be shown together + with the error message returned by the device. Sometimes the text + message is not an actual error. It could be a warning that should be + ignored. The NED has a static list of known warnings, presently these: + // general + "is in use", + "already exists", + "not completed its shutdown" + If you stumble upon a warning not in the list above, which is quite + likely due to the large number of warnings or some customers are interested + in aborting commit if NED receives those warnings from device, you can configure the + NED to ignore them under ned-settings in the alu-isam-config-warning + list. The list resides in two places and you can configure a + warning in any of these: + devices device ned-settings alu-isam alu-isam-config-warning + devices global-settings ned-settings alu-isam alu-isam-config-warning + alu-isam-config-warning is a regular expression string list of warnings that should also be ignored. + For example, to add a new warning exception: + admin@ncs(config)# devices device ned-settings alu-isam alu-isam-config-warning ".*when SN is bogus or NULL.*" + admin@ncs(config-device-isam-1)# commit + Commit complete. + admin@ncs(config-device-isam-1)# disconnect + admin@ncs(config-device-isam-1)# connect + result true + + Note that in order for the warning exception to take effect, you must disconnect and connect again. + + +# 8. ned-settings alu-isam alu-isam-extra-errors +------------------------------------------------ + + This section describes device messages that must be handled as errors. + + - alu-isam alu-isam-extra-errors + + - message + + Extra error regular expresio, e.g. ^INFO:.*. + + Depending on the use case, sometimes different messages from the device must be threated as errors + even if they are not error messages. + + For example, to add a new extra error message: + + admin@ncs(config)# devices device ned-settings alu-isam alu-isam-extra-errors "INFO: PIP #1370.*" + admin@ncs(config-device-isam-1)# commit + Commit complete. + admin@ncs(config-device-isam-1)# disconnect + admin@ncs(config-device-isam-1)# connect + result true + + Note that in order for the new error messages to take effect, you must disconnect and connect again. + + +# 9. ned-settings alu-isam connection +------------------------------------- + + Configure settings specific to the connection between NED and device. + + + - connection number-of-retries (default 0) + + Configure max number of retries the NED will try to connect to the device before giving up. + Default 0. + + + - connection time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + +# 10. ned-settings alu-isam logger +---------------------------------- + + Settings for controlling logs generated. + + + - logger level (default info) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - logger java (default false) + + Toggle logs to be added to ncs-java-vm.log. + + +# 11. ned-settings alu-isam proxy +--------------------------------- + + Configure NED to access device via a proxy. + + + - proxy remote-connection + + Connection type between proxy and device. + + ssh - SSH jump host proxy. + + telnet - TELNET jump host proxy. + + serial - Terminal server proxy. + + + - proxy remote-address + + Address of host behind the proxy. + + + - proxy remote-port + + Port of host behind the proxy. + + + - proxy remote-command + + Connection command used to initiate proxy on device. Optional for ssh/telnet. Accepts + $(proxy/remote-xxx) for inserting remote-xxx config. + + + - proxy remote-name + + User name on the device behind the proxy. + + + - proxy remote-password + + Password on the device behind the proxy. + + + - proxy authgroup + + Authentication credentials for the device behind the proxy. + + + - proxy proxy-prompt + + Prompt pattern on the proxy host. + + + - proxy remote-ssh-args + + Additional arguments used to establish proxy connection (appended to end of ssh cmd line). + + Do as follows to setup to connect to an ISAM device that resides + behind a proxy or terminal server: + + +-----+ A +-------+ B +------+ + | NCS | <--> | proxy | <--> | ISAM | + +-----+ +-------+ +------+ + + Setup connection (A): + + # devices device isam0 address + # devices device isam0 port + # devices device isam0 device-type cli protocol + # devices authgroups group isamgroup umap admin remote-name + # devices authgroups group isamgroup umap admin remote-password + # devices device isam0 authgroup isamgroup + + Setup connection (B): + + Define the type of connection to the device. The type 'serial' is + used for terminal servers: + + # devices device isam0 ned-settings alu-isam proxy remote-connection + + Define login credentials for the device: + + # devices device isam0 ned-settings alu-isam proxy remote-name + # devices device isam0 ned-settings alu-isam proxy remote-password + + If remote-connection is ssh or telnet then configure the following as well: + + # devices device isam0 ned-settings alu-isam proxy proxy-prompt + # devices device isam0 ned-settings alu-isam proxy remote-address
+ # devices device isam0 ned-settings alu-isam proxy remote-port + # commit + # disconnect + # connect + + diff --git a/alu-isam/README.md b/alu-isam/README.md new file mode 100644 index 00000000..0f27b83d --- /dev/null +++ b/alu-isam/README.md @@ -0,0 +1,810 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + ``` + + +# 1. General +------------ + + This document describes the alu-isam NED. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | See README.md for further info on how to configure the NED to | + | | | use netsim as a RESTCONF target. | + | | | | + | check-sync | yes | | + | | | | + | partial-sync-from | yes | | + | | | | + | live-status actions | no | | + | | | | + | live-status show | yes | | + | | | | + | load-native-config | no | | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | ALU 7356 iSAM | R5.0 | | | + | | | | | + | nfxs-e | R6.2.04sd | | | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--alu-isam-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-alu-isam-1.0.1.signed.bin + > ./ncs-6.0-alu-isam-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-alu-isam-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-alu-isam-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `alu-isam-.`: + + ``` + > tar xfz ncs-6.0-alu-isam-1.0.1.tar.gz + > ls -d */ + alu-isam-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package alu-isam-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package alu-isam-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-alu-isam-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package alu-isam-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/alu-isam-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-alu-isam-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-alu-isam-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install alu-isam-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-alu-isam-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-alu-isam-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id alu-isam-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-alu-isam-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings alu-isam logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings alu-isam logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.isam \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings alu-isam logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings alu-isam logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + For instance, create a new vlan: + + # configure + # devices device isam0 config + # vlan + # id 1234 + # name TESTVLAN + # commit + + Verify that NCS is in-sync with the device + + Alternative 1 + # check-sync + + Alternative 2 + # compare-config + + +# 5. Built in live-status actions +--------------------------------- + + NONE + + +# 6. Built in live-status show +------------------------------ + + The NED has support for a subset of isam's native operational commands residing + under device live-status. Presently, the following commands are supported: + + - info Execute info commands + - show Execute show commands + + For example, to display the boot settings of the ALU device: + + admin@ncs# devices device isam-1 live-status exec show system shub entry version + result + ================================================================================ + version table + ================================================================================ + sw-release-num : LANX_R5.0.2.04 + ================================================================================ + + +# 7. Limitations +---------------- + + 7.1 Auto-config: + ---------------- + What happens when /equipment/slot/planned-type/ value is + modified from "fwlt-b" to "fglt-b": + Note: Currently only considers modeled nodes/leaves. + + In /qos/: + All chpair interfaces related to the specific slot is removed. [1-8] + pon interfaces are created, with relation to the specific slot [1-16] + In /interface/: + All chpair ports related to the specific slot is removed. [1-8] + pon ports are created, with relation to the specific slot [1-16] + In /pon/: + interfaces are created, with relation to the specific slot [1-16] + + When changing /equipment/slot/planned-type/ from "fglt-b" to "fwlt-b": + (Reverse of above, reverse result) + In /qos/: + All pon interfaces related to the specific slot is removed. [1-16] + chpair interfaces are created, with relation to the specific slot [1-8] + In /interface/: + All pon ports related to the specific slot is removed. [1-16] + chpair ports are created, with relation to the specific slot [1-8] + In /pon/: + interfaces are removed, with relation to the specific slot [1-16] + + 7.2 admin-state unlocked in voice/ont/voice-port : + ------------------------------------------------- + + As the admin-state unlocked is dependent for creating other leafs in voice/ont/voice-port and voice/ont/voice-sip-port. + + Before changing the values of the leafs inside list voice-port and voice-sip-port, configure "no admin-state" in voice/ont/voice-port which will be translated to admin-state locked like below + + admin@ncs(config-voice-port-1/1/1/1/1/1/1)# no admin-state + admin@ncs(config-voice-port-1/1/1/1/1/1/1)# pots-pwr-timer 600 + admin@ncs(config)# commit dry-run outformat native + native { + device { + name isam-1 + data exit all + configure + voice + ont + voice-port 1/1/1/1/1/1/1 + admin-state locked + pots-pwr-timer 600 + exit + voice-sip-port 1/1/1/1/1/1/1 + user-aor test2 + exit + exit + exit + exit all + } + } + admin@ncs(config)# commit + Commit complete. + admin@ncs(config)# top rollback configuration + admin@ncs(config)# commit dry-run outformat native + native { + device { + name isam-1 + data exit all + configure + voice + ont + voice-port 1/1/1/1/1/1/1 + pots-pwr-timer 300 + exit + voice-sip-port 1/1/1/1/1/1/1 + user-aor test1 + exit + voice-port 1/1/1/1/1/1/1 + admin-state unlocked + exit + exit + exit + exit all + } + } + admin@ncs(config)# commit + Commit complete. + admin@ncs(config)# + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings alu-isam logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings alu-isam logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` diff --git a/alu-omniswitch-6k/README.md b/alu-omniswitch-6k/README.md new file mode 100644 index 00000000..f10f0407 --- /dev/null +++ b/alu-omniswitch-6k/README.md @@ -0,0 +1,962 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + 11. Save configuration on the device (persistent after restart) + 12. Adding known errors to a list that are checked at "connect" and "sync-from" + ``` + + +# 1. General +------------ + + This document describes the alu-omniswitch-6k NED. + + The NED has been tested with ALU Omniswitch 6850 and ALU Omniswitch 6860 devices. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | - | + | | | | + | check-sync | yes | - | + | | | | + | partial-sync-from | yes | - | + | | | | + | live-status actions | yes | - | + | | | | + | live-status show | yes | - | + | | | | + | load-native-config | yes | - | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | ALCATEL OMNISWITCH 6850 | 6.4.4.411.R01 | Alcate | - | + | | | l- | | + | | | Lucent | | + | | | OS6850 | | + | | | -U24X | | + | | | | | + | ALCATEL OMNISWITCH 6860 | 8.2.1.276.R01 | Alcate | - | + | | | l- | | + | | | Lucent | | + | | | OS6860 | | + | | | E-U28 | | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--alu-omniswitch-6k-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-alu-omniswitch-6k-1.0.1.signed.bin + > ./ncs-6.0-alu-omniswitch-6k-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-alu-omniswitch-6k-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-alu-omniswitch-6k-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `alu-omniswitch-6k-.`: + + ``` + > tar xfz ncs-6.0-alu-omniswitch-6k-1.0.1.tar.gz + > ls -d */ + alu-omniswitch-6k-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package alu-omniswitch-6k-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package alu-omniswitch-6k-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-alu-omniswitch-6k-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package alu-omniswitch-6k-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/alu-omniswitch-6k-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-alu-omniswitch-6k-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-alu-omniswitch-6k-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install alu-omniswitch-6k-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-alu-omniswitch-6k-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-alu-omniswitch-6k-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id alu-omniswitch-6k-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-alu-omniswitch-6k-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings alu-omniswitch-6k logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings alu-omniswitch-6k logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.aluomniswitch6k \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings alu-omniswitch-6k logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings alu-omniswitch-6k logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + **SSHJ DEBUG LOGGING** + For issues related to the ssh connection it is often useful to enable full logging in the SSHJ ssh client. + This will make SSHJ print additional log entries in `$NSO_RUNDIR/logs/ncs-java-vm.log`: + +``` +admin@ncs(config)# java-vm java-logging logger net.schmizz.sshj level level-all +admin@ncs(config)# commit +``` + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + Create a vlan and a new ip interface: + + ``` + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices device dev-1 config + admin@ncs(config-config)# vlan 17 enable name VLAN17 + admin@ncs(config-config)# ip interface "TEST" address 1.2.3.4 mask 255.255.255.0 vlan 17 no-forward admin enable + admin@ncs(config-config)# commit + Commit complete. + ``` + + Verify that NCS is in-sync with the device + + Alternative 1 + + ``` + admin@ncs(config-config)# check-sync + result in-sync + ``` + + Alternative 2 + + ``` + admin@ncs(config-config)# compare-config + admin@ncs(config-config)# + ``` + + +# 5. Built in live-status actions +--------------------------------- + + Example: + + ``` + admin@ncs(config)# devices device dev-1 live-status exec any show system + result + System: + Description: 6.3.4.378.R01 GA, March 11, 2009., + Object ID: 1.3.6.1.4.1.6486.800.1.1.2.1.7.1.3, + Up Time: 5 days 11 hours 29 minutes and 57 seconds, + Contact: NONE, + Name: alu6850, + Location: NONE, + Services: 72, + Date & Time: SUN JUN 04 2022 22:40:40 (PDT) + + Flash Space: + Primary CMM: + Available (bytes): 10492928, + Comments : None + ``` + + +# 6. Built in live-status show +------------------------------ + + Example: + + ``` + admin@ncs# show devices device alu-device live-status interfaces + SLOT ADMIN LINK + PORT STATUS STATUS ALIAS + --------------------------------------------------------------------------------------------------------------- + 1/1 disable down "" + 1/2 disable down "" + 1/3 disable down "" + 1/4 disable down "" + 1/5 enable down "" + 1/6 disable down "" + 1/7 disable down "" + 1/8 enable down "MiTOP DS3 Circuit Emulation|Drake_05-15- + 13" + 1/9 disable down "" + 1/10 enable down "test|12-31-2012|Drake's testing" + 1/11 disable down "" + 1/12 disable down "" + 1/13 disable down "" + 1/14 disable down "test2|06-30-13|Strategic IP" + 1/15 disable down "test2|06-30-13|Strategic IP" + 1/16 enable down "Joe" + 1/17 enable down "" + 1/18 enable down "" + 1/19 enable down "3.3" + 1/20 enable down "" + 1/21 disable down "" + 1/22 enable down " " + 1/23 enable down "test3|05-15-13" + 1/24 enable up "" + 1/25 enable down "IL|12-31-15|alu6850-6|1/26|ESP<-->MES Ri + ng" + 1/26 enable down "IL|12-31-15|alu7750-1|1/1/1|ESP<-->MES R + ing" + ``` + + +# 7. Limitations +---------------- + + NONE + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings alu-omniswitch-6k logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings alu-omniswitch-6k logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` + +# 11. Save configuration on the device (persistent after restart) +---------------------------------------------------------------- + +A. The user can save the configuration on the device at a certain moment (and if the contents of the "working directory" have been verified as the best version of the CMM files). +Depending on the type of the switch (standalone or part of the stack), the user can issue the following commands: + +For a standalone device +``` +admin@ncs# devices device dev-1 live-status exec copy working certified +``` + +For a standalone device + +``` +admin@ncs# devices device dev-1 live-status exec copy flash-synchro +``` + +Please note that if the user is using `commit-queue`, then he must assure that all the commits have been executed and there is no other task in the queue. Please see the following example. + +Commands from the first commit: + +``` +admin@ncs(config-config)# system contact NONE +admin@ncs(config-config)# system location NONE +admin@ncs(config-config)# no system timezone + +admin@ncs(config-config)# commit dry-run outformat native +native { + device { + name dev-1 + data system contact NONE + system location NONE + system timezone PST + qos apply + } +} + +admin@ncs(config-config)# commit commit-queue async +commit-queue { + id 1548246954737 + status async +} +Commit complete. +``` + +Command for the second commit: + +``` +admin@ncs(config-config)# session timeout ftp 5 +admin@ncs(config-config)# commit commit-queue async +commit-queue { + id 1548246957707 + status async +} +Commit complete. +``` + +Checking the status of the commits: + +``` +admin@ncs(config-config)# end +admin@ncs# show devices commit-queue | notab +devices commit-queue queue-item 1548246954737 + age 6 + status executing + devices [ dev-1 ] + is-atomic true + +devices commit-queue queue-item 1548246957707 + age 3 + status blocked + devices [ dev-1 ] + waiting-for [ dev-1 ] + is-atomic true +``` + +At this moment we have 2 commits in the commit-queue, with 2 commit IDs. +Before sending any "copy" command to the device, the user must wait for all the commands to be executed (no commits in the queue). +Otherwise, a different device configuration can be saved on the "certified" directory that the one we want to(the "copy" command executed before the commit). + +To check if there are commands pending, please use the following command: + +``` +admin@ncs# show devices commit-queue | notab +devices commit-queue queue-item 1548246957707 + age 15 + status executing + devices [ dev-1 ] + is-atomic true +admin@ncs# show devices commit-queue | notab +devices commit-queue queue-item 1548246957707 + age 17 + status executing + devices [ dev-1 ] + is-atomic true +admin@ncs# show devices commit-queue | notab +devices commit-queue queue-item 1548246957707 + age 21 + status executing + devices [ dev-1 ] + is-atomic true + +admin@ncs# show devices commit-queue | notab +% No entries found. +``` + +If there are no more entries, then it is safe for the user to send a "copy" exec action to the device, as below: + +``` +admin@ncs# devices device dev-1 live-status exec copy working certified +``` + +B. Also, these commands (the "copy" commands) can be triggered automatically from the NED, if "copy-certified" is enabled under ned-settings: + +``` +admin@ncs# config +admin@ncs(config)# devices device dev-1 ned-settings alu-omniswitch-6k persistent-store copy-certified enabled +``` + +Note: +- `disconnect` and `connect` commands are mandatory for enable a ned-setting + + +# 12. Adding known errors to a list that are checked at "connect" and "sync-from" +--------------------------------------------------------------------------------- + +Sometimes, errors could appear even for basic commands, like: "show system" (called at `connect`) or when NED wants to perform a `sync-from`. +Such an example is the case when the user has limited access towards device and the "show system" command will return an error similar to: "ERROR: Authorization failed. No functional privileges for this command". + +The above type of error is handled in the NED, but if similar errors occur, the user can add those to a list +that will be treated in NED. To be able to do that, the user must configure "alu-omniswitch-6k/write/known-errors" +list under the `ned-settings` section. + +Example: +Supposing that "show system" might return, in particular cases, the following error: +"Error: An error message". +To make the NED aware of checking this error at `connect` - where "show system" command is called - the +following setting is necessary: + +``` +admin@ncs# config +Entering configuration mode terminal +admin@ncs(config)# devices device dev-1 ned-settings alu-omniswitch-6k write known-errors ? +Possible completions: + Warning regular expression, part of the device's reply +admin@ncs(config)# devices device dev-1 ned-settings alu-omniswitch-6k write known-errors "(?m)^Error: An error message" +admin@ncs(config-device-dev-1)# commit +Commit complete. + +admin@ncs(config-device-dev-1)# config +admin@ncs(config-config)# disconnect +admin@ncs(config-config)# connect +``` + +Note: +- `disconnect` and `connect` commands are mandatory above, so the ned-setting can be taken into account +- special characters from regular expressions must be escaped using \"\\" character + +The value for "known-errors" list must be a regular expression. In the above example, the exact error is matched. +But if the following regular expression: "(?m)^Error:.*", is used instead of the previous, then NED will +throw an exception if the reply from the device (for "show-system" command) has a line starting with string "Error:". diff --git a/alu-sr/README-ned-settings.md b/alu-sr/README-ned-settings.md new file mode 100644 index 00000000..90930d79 --- /dev/null +++ b/alu-sr/README-ned-settings.md @@ -0,0 +1,795 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/alu-sr/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/alu-sr/ + device + /ncs:/device/devices/device:/ned-settings/alu-sr/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings alu-sr + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings alu-sr + 2. persistent-store + 3. meta-data + 4. logger + 5. proxy + 6. proxy2 + 7. live-status + 7.1. auto-prompts + 8. developer + 9. connection + 10. get-device-config-settings + 10.1. detailed-config + 11. apply-device-config-settings + 12. behaviour + 13. transaction + 13.1. config-abort-warning + 13.2. config-ignore-error + 14. console + ``` + + +# 1. ned-settings alu-sr +------------------------ + + + - trans-id-method (default config-data) + + Configure how the NED shall calculate the transaction id. Typically used after each commit and + for check-sync operations. + + config-hash - Use a snapshot of the whole device running config + for calculation. + + rollback-timestamp - Use the time stamp of the latest rollback checkpoint + for calculation. The system rollback feature must be + on. + + last-modified-timestamp - Use the 'time last modified' time stamp generated by + the ALU device for calculation. Note, this time + stamp is not available on all ALU devices. See + README. + + last-saved-timestamp - Use the 'time last saved' time stamp generated by + the ALU device for calculation. Note, this method is + not reliable. See README. + + last-modified-and-last-saved-timestamp - Use the 'time last modified' and 'time last saved' + time stamp generated by the ALU device for + calculation. Note, this method is not reliable. See + README. + + config-data - Use a snapshot of the device running config filtered + through NED yang schema - eg removes device data + that it is not modeled by the NED. + + + - candidate-commit (default disabled) + + Make the NED use the candidate commit feature available on some ALU devices. + + disabled - Apply configuration the standard way, one-by-one. (default). + + enabled - Apply configuration into a candidate and then commit it.The candidate feature must + be enabled on the device. + + + - extended-parser (default auto) + + Make the alu-sr NED enable extensions to ease the task of the NSO CLI command parser. A common + problem with this parser is that it can easily get lost when trying to parse ALU configuration + not supported by the YANG model. In particular it is very sensitive for unsupported config + that generates a mode switch. + + disabled - Load configuration the standard way. + + robust-mode - The configuration dump is run through a pre-parser which is cleaning it from + all elements currently not supported in the YANG model (default). + + turbo-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO using a + Maapi SetValues() call. + + turbo-xml-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO in XML + format. + + auto - Uses turbo-mode when available, will use fastest availablemethod to load + data to NSO. If NSO doesn't support data-loading from CLI NED, robust-mode + is used. + + + - number-of-lines-to-send-in-chunk (default 100) + + Number of commands lines in a chunk sent by the alu-sr NED to the device. Default is 100. A + higher number normally result in better performance but will also have negative impact on the + error handling. This command does also control the chunk sizes used in partial show bulk + mode. + + + - partial-show-method (default bulk-mode) + + Method to use when executing a partial show on the device (for instance when doing a 'commit + no-overwrite'). + + bulk-mode - The NED executes in a bulk mode fashion. Commands to display config on the + requested locations on the device are sent in chunks to minimize the round trip + time. The result is gathered when all commands have been sent to the device. + (default). + + walk-mode - The NED walks the config tree on the device step by step and extracts the + config on the requested locations. This method is much slower than bulk mode + but can be an alternative for devices that are not able to handle chunks of + commands properly. + + filter-mode - The NED fetches a full configuration dump from the device. It then filters out + everything except the requested the parts. The filtered config is then sent + back to NSO. +Note, this method is always used when connected to a NETSIM + device. + + +# 2. ned-settings alu-sr persistent-store +----------------------------------------- + + Configure how the NED shall save configuration to persistent memory. + + + - persistent-store admin-save (default on-persist) + + Configure how and when an applied config is saved to persistent memory on the device. + + on-commit - Save configuration immediately after the config has been successfully applied on + the device. If an error occurs when saving the whole running config will be + rolled back. + + on-persist - Save configuration during the NED persist handler. Called after the config has + been successfully applied and commited If an error occurs when saving an alarm + will be triggered. No rollback of the running config is done. (default). + + disabled - Disable saving the applied config to persistent memory. + + + - persistent-store file-url + + Configure a file/URL to save config in. If not set, system default will be used. + + +# 3. ned-settings alu-sr meta-data +---------------------------------- + + Configure how the NED shall operate on the meta-data / cli extension tags used in the YANG model. + + + - meta-data shutdown-before-set (default enabled) + + This operation makes the NED shutdown a parent before (given by path) setting the value of + this leaf, Default: enabled. + + enabled - enabled. + + disabled - disabled. + + + - meta-data redeploy (default enabled) + + This operation makes the NED redeploy another leaf (given by path) after setting the value of + this leaf, Default: enabled. + + enabled - enabled. + + disabled - disabled. + + + - meta-data re-enter-authentication-key (default auto) + + Configure the NED to support re-entering of the key value when configuring auth keys on + certain router protos. Only relevant on devices running TiMOS 15 or later and booted in + FIPS-140-2 mode, Default: auto. + + enabled - enabled. + + disabled - disabled. + + auto - auto. + + + - meta-data check-ospf-area-id (default enabled) + + This operation makes the NED throw if an OSPF area id is configured in integer format, since + it will otherwise cause an out of sync issue with the device. The OSPF area id shall always be + specified in ipv4 format. This is how it will be displayed by the device. + + enabled - enabled. + + disabled - disabled. + + + - meta-data check-service-id (default enabled) + + This operation makes the NED throw if a string formatted service id is used for a + vpls/vrpn/epipe/cpipe service. When configuring services through NSO the numeric id shall + always be used to avoid out-of-sync issues with the device. + + enabled - enabled. + + disabled - disabled. + + + - meta-data bgp-neighbor-wait-after-delete (default 0) + + This setting allows for setting a delay in seconds after deletion of a BGP neighbor entry + (default 0). Delete operations of neighbor entries can take some time to execute by the ALU + device. Some use cases require a small delay after the deletion. For instance when moving a + neighbor from one group to another. + + + - meta-data force-no-filter-name (default disabled) + + This setting forces the absence of /filter/[ip|ipv6|mac]-filter/name which normally is present + in SR-OS >= 15.1 but is absent on certain models/versions. + + enabled - enabled. + + disabled - disabled. + + +# 4. ned-settings alu-sr logger +------------------------------- + + Settings for controlling logs generated. + + + - logger level (default info) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - logger java (default true) + + Toggle logs to be added to ncs-java-vm.log. + + +# 5. ned-settings alu-sr proxy +------------------------------ + + Configure NED to access device via a proxy. + + + - proxy remote-connection + + Connection type between proxy and device. + + ssh - ssh. + + telnet - telnet. + + serial - serial. + + + - proxy remote-address + + Address of host behind the proxy. + + + - proxy remote-port + + Port of host behind the proxy. + + + - proxy remote-name + + User name on the device behind the proxy. + + + - proxy remote-password + + Password on the device behind the proxy. + + + - proxy remote-secondary-password + + Enable password on the device behind the proxy. + + + - proxy authgroup + + Authentication credentials for the device behind the proxy. + + + - proxy proxy-prompt + + Prompt pattern on the proxy host. + + + - proxy remote-ssh-args + + Additional arguments used to establish proxy connection. + + +# 6. ned-settings alu-sr proxy2 +------------------------------- + + Configure NED to access device via a proxy2. + + + - proxy2 remote-connection + + Connection type between proxy and device. + + ssh - ssh. + + telnet - telnet. + + serial - serial. + + + - proxy2 remote-address + + Address of host behind the proxy. + + + - proxy2 remote-port + + Port of host behind the proxy. + + + - proxy2 remote-name + + User name on the device behind the proxy. + + + - proxy2 remote-password + + Password on the device behind the proxy. + + + - proxy2 remote-secondary-password + + Enable password on the device behind the proxy. + + + - proxy2 authgroup + + Authentication credentials for the device behind the proxy. + + + - proxy2 proxy-prompt + + Prompt pattern on the proxy host. + + + - proxy2 remote-ssh-args + + Additional arguments used to establish proxy connection. + + +# 7. ned-settings alu-sr live-status +------------------------------------ + + Configure NED settings related to live-status. + + + - live-status time-to-live (default 50) + + Define time-to-live for data fetched from the device via live-status.(default 50). + + +## 7.1. ned-settings alu-sr live-status auto-prompts +---------------------------------------------------- + + Pre-stored answers to device prompting questions. + + - live-status auto-prompts + + - id + + List id, any string. + + - question + + Device question, regular expression. + + - answer + + Answer to device question. + + +# 8. ned-settings alu-sr developer +---------------------------------- + + Contains settings used by the NED developers. + + + - developer load-from-file + + Make the NED load a file containing raw device config when doing sync-from. Does only work on + NETSIM targets. + + + - developer load-native-config allow-delete (default false) + + Enable this setting to be able to handle limited delete operations with 'load-native-config'. + Please note that not all syntax available on a real device works, some delete operations can + not be parsed by the NED. Use the 'verbose' flag to 'load-native-config' to see if delete + commands can be parsed. Currently this is only supported when 'extended-parser' is set to + 'turbo-xml-mode'. + + + - developer load-native-config delete-with-remove (default false) + + Enable this setting to use 'remove' instead of 'delete' when sending delete operations to NSO. + This is useful when doing delete commands for data that might not be present in CDB. Please + note that deletes for missing data will still be part of transaction, and will be sent to + device. Use with care, and do proper testing to understand behaviour. + + + - developer platform model + + Override device model name/number. + + + - developer platform name + + Override device name. + + + - developer platform version + + Override device version. + + + - developer device-type (default netsim) + + Real or simulated device. + + netsim - netsim. + + device - device. + + + - developer progress-verbosity (default disabled) + + Maximum NED verbosity level which will get written in devel.log file. + + disabled - disabled. + + normal - normal. + + verbose - verbose. + + very-verbose - very-verbose. + + debug - debug. + + + - developer read-only-bof (default disabled) + + When this setting is enabled the bof config will be read-only. + + enabled - enabled. + + disabled - disabled. + + + - developer filter-password-emit (default disabled) + + When this setting is enabled the system security user password are filtered when the NED + applies config to the device. + + enabled - enabled. + + disabled - disabled. + + + - developer trace-connection (default false) + + Enable developer connection tracing. WARNING: may choke NSO with large printouts. + + +# 9. ned-settings alu-sr connection +----------------------------------- + + Configure settings specific to the connection between NED and device. + + + - connection ssh client + + Configure the SSH client to use. Relevant only when using the NED with NSO 5.6 or later. + + ganymed - The legacy SSH client. Used on all older versions of NSO. + + sshj - The new SSH client. Supports the latest key algorithms etc. This is the default + when using the NED on NSO 5.6 or later. + + + - connection ssh host-key known-hosts-file + + Path to openssh formatted 'known_hosts' file containing valid host keys. + + + - connection ssh host-key public-key-file + + Path to openssh formatted public (.pub) host key file. + + + - connection ssh auth-key private-key-file + + Path to openssh formatted private key file. + + + - connection connector + + Change the default connector. Default 'ned-connector-default.json'. + + + - connection number-of-retries (default 0) + + Configure max number of retries the NED will try to connect to the device before giving up. + Default 0. + + + - connection time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + + - connection terminal println-mode (default default) + + default - System property line.separator default. + + ocrnl - Translate carriage return to newline. + + onocr - Translate newline to carriage return-newline. + + onlret - Newline performs a carriage return. + + +# 10. ned-settings alu-sr get-device-config-settings +---------------------------------------------------- + + Configure how the NED shall read config from the device. + + + - get-device-config-settings method (default cli) + + The method to use. + + cli - Dump the running config using the 'admin display-config' CLI command + (default). + + sftp-transfer - Transfer the latest saved config from the device via the SFTP protocol. Note + that this method is not reliable. The latest saved config is not necessarily + the same as the running config. A NED configured for this will not be able to + detect out of band changes that have not been saved to file yet.Supported by + newer TiMOS versions. + + scp-transfer - Transfer the latest saved config from the device via the SCP protocol. Note + that this method is not reliable. The latest saved config is not necessarily + the same as the running config. A NED configured for this will not be able to + detect out of band changes that have not been saved to file yet. Supported by + older TiMOS versions. + + + - get-device-config-settings file + + The name of the file containing latest saved config. + + + - get-device-config-settings save-running-config-first + + Make the NED automatically save the running config to the specified file before starting the + SFTP/SCP transfer. + + enabled - enabled. + + disabled - disabled. + + + - get-device-config-settings extra get-port-speed (default disabled) + + Makes the NED do an additional fetch of the speed setting on each port on the device. The port + speed is a setting that is trimmed by the device when set to default. This feature generates + one extra config dump, which might affect sync-from performance. The command used does not + work on TiMOS versions < 10. The feature is not used when connected to NETSIM. + + enabled - enabled. + + disabled - disabled. + + + - get-device-config-settings extra get-port-type (default disabled) + + Makes the NED do an additional fetch of the port type on each port on the device. + + enabled - enabled. + + disabled - disabled. + + + - get-device-config-settings extra get-router-sgt-qos (default disabled) + + Enable this flag to pull default values of router/sgt-qos. + + enabled - enabled. + + disabled - disabled. + + + - get-device-config-settings extra enable-detailed-config (default true) + + Enable/Disable fetching of detailed nodes configuration. NOTE: enabling this feature may lead + to sync-from time increase in case of big device configuration. In such case, read-timeut may + be needed to be increased. + + +## 10.1. ned-settings alu-sr get-device-config-settings extra detailed-config +----------------------------------------------------------------------------- + + This is a list containing device config that needs to be fetched with detailed info (eg include + device hidden defaults in the output). Note: each list entry is a regex pattern that will be used + in the device 'admin display-config detail | match ' expression (regex to be tested on a + real device to properly match the needed config). + + - get-device-config-settings extra detailed-config + + - regex + + +# 11. ned-settings alu-sr apply-device-config-settings +------------------------------------------------------ + + Configure how the NED shall write config to the device. + + + - apply-device-config-settings method (default cli) + + The method to use. + + cli - Configure through the CLI (default). + + sftp-transfer - Transfer as file to the device via the SFTP protocol. Then apply the config + using the 'exec' command on the device.Supported by newer versions of TiMOS. + + scp-transfer - Transfer as file to the device via the SCP protocol. Then apply the config + using the 'exec' command on the device.Supported by older versions of TiMOS. + + + - apply-device-config-settings file (default cf3:/alu-sr-tmp.cfg) + + The name of the temporary file to use when transferring the config (default: + 'cf3:alu-sr-tmp.cfg'. + + +# 12. ned-settings alu-sr behaviour +----------------------------------- + + NED specific behaviours. + + +# 13. ned-settings alu-sr transaction +------------------------------------- + + + - transaction ignore-bof-in-trans-id + + Configure if BOF section should be ignored or not at trans-id computation (eg ignore BOF at + check-sync). + + enabled - enabled. + + disabled - disabled. + + + - transaction cleartext-provisioning (default enabled) + + Enable this to allow for cleartext key/passwords provisioning without going out-of-sync(i.e. + where device obfuscates/encrypts value in running-config). + + enabled - enabled. + + disabled - disabled. + + + - transaction cleartext-stored-encrypted (default disabled) + + When 'cleartext-provisioning' is enabled, enable this setting to enforce keys/passwords CDB + storedvalues to be encrypted using NSO's built in encryption types (e.g. + 'tailf:aes-cfb-128-encrypted-string').NOTE: the type of the values in the yang-model of alu-sr + is NOT the encrypted type by default, it is still plain 'string'. However, the service + template/code used to set the values must use an encrypted type.The NED can be instructed to + use tailf:aes-cfb-128-encrypted-string for passwords by default and hence to do + auto-encryption of the passwords, but this requires to recompile the NED with + NEDCOM_SECRET_TYPE flag set (eg 'make NEDCOM_SECRET_TYPE="tailf:aes-cfb-128-encrypted-string" + clean all'). + + enabled - enabled. + + disabled - disabled. + + + - transaction abort-on-diff (default false) + + Enable to detect diff immediately when config is applied (i.e. in commit/abort/revert). + If a diff is detected an exception is thrown, having the effect in commit that the transaction + is aborted (showing the diff). Note that this means some overhead in commit, where whole config + needs to be retrieved from device to do compare. It's a more exact way to detect OOB changes, + silently dropped config, and unknown side-effects to config (i.e. all causing a diff compared + to NSO state). In fact, it's the only method which guarantees that the config was actually + applied as desired. Since this incurs overhead it is strongly adviced that this feature is + only used during development. + + +## 13.1. ned-settings alu-sr transaction config-abort-warning +------------------------------------------------------------- + + Configure additional device warnings that shall be treated as errors and trigger an abort in the + NED. Enter as a regex. + + - transaction config-abort-warning + + - warning + + Warning regular expression, e.g. '.*interface.* does not exist.*' . NOTE: please use + lowercase letters for device warnings, except for regex keywords. + + +## 13.2. ned-settings alu-sr transaction config-ignore-error +------------------------------------------------------------ + + Configure additional device errors that shall be treated as warnings (i.e. to be ignored, not + aborting transaction). + + - transaction config-ignore-error + + - error + + Warning regular expression, e.g. '^.*interface.* does not exist.*$'. NOTE: please use + lowercase letters for device warnings, except for regex keywords. + + +# 14. ned-settings alu-sr console +--------------------------------- + + Settings used while interacting with a device. + + + - console line-feed + + Overwrites default line-feed character. + + diff --git a/alu-sr/README.md b/alu-sr/README.md new file mode 100644 index 00000000..78cef467 --- /dev/null +++ b/alu-sr/README.md @@ -0,0 +1,1306 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + 11. NED Secrets - Securing your Secrets + ``` + + +# 1. General +------------ + + This document describes the alu-sr NED. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | Default emulated device: 7750 SROS 14.0.0 | + | | | | + | check-sync | yes | Several check-sync strategies acceped (config-hash, last-updated | + | | | timestamp etc) | + | | | | + | partial-sync-from | yes | Native device CLI commands supported (eg 'info' inside a node). | + | | | Check partial-show-method ned-settings | + | | | | + | live-status actions | yes | Commands suported as live-status exec: admin|any|debug|ping|show | + | | | | + | live-status show | yes | Supports several live-status 'show' TTL-based commands | + | | | | + | load-native-config | yes | | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + Custom NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | apply-device-config- | yes | Supported methods: CLI, scp-transfer, sftp-transfer | + | method | | | + | | | | + | apply-device-config- | yes | Supported methods: CLI, scp-transfer, sftp-transfer | + | method | | | + | | | | + | get-device-config-method | yes | Supported methods: CLI, scp-transfer, sftp-transfer | + | | | | + | candidate-commit | yes | Needs to be enabled on the device also | + | | | | + | number-of-lines-to-send- | yes | Configurable number of lines to be sent in one chunk | + | in-chunk | | | + | | | | + | persistent-store | yes | Configure how the NED shall save configuration to persistent | + | | | memory | + | | | | + | trans-id-method | yes | Configure how the NED shall calculate the transaction id (full | + | | | config hash, last modified date etc) | + | | | | + | ned-secrets | yes | NED supports device-enctypted password caching. Please check | + | | | README.md | + | | | | + | proxy | yes | Supports up to two proxy jumps | + | | | | + | custom-transaction-error- | yes | Supports definition of custom device errors to be thrown by the | + | definition | | NED | + | | | | + | ignore-bof-in-trans-id | yes | Configure if BOF section should be ignored or not at trans-id | + | | | computation (eg ignore BOF at check-sync) | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | ALCATEL SR 7750 | C-12.0.R8 | SROS | 7750 SROS-C-12.0.R8 | + | | | | | + | ALCATEL SR 7750 | C-14.0.R4 | SROS | 7750 SROS-C-14.0.R4 | + | | | | | + | ALCATEL SR 7750 | C-15.1.R2 | SROS | 7750 SROS-C-15.1.R2 | + | | | | | + | ALCATEL SR 7750 | C-16.0.R3 | SROS | 7750 SROS-C-16.0.R3 | + | | | | | + | ALCATEL SR 7750 | B-21.5.R2 | SROS | 7750 SROS-B-21.5.R2 | + | | | | | + | ALCATEL SR 7950 | C-19.7.R2 | SROS | 7950 SROS-C-19.7.R2 | + | | | | | + | ALCATEL SR 7250 | C-19.10.R6 | SROS | 7250 SROS-C-19.10.R6 | + | | | | | + | ALCATEL SR 7705 | B-5.0.R3 | SROS | 7705 SROS-B-5.0.R3 | + | | | | | + | ALCATEL SR 7210 | B-4.0.R6 | SROS | 7210 SROS-B-4.0.R6 | + | | | | | + | ALCATEL SR 7210 | B-3.0.R3 | SROS | 7210 SROS-B-3.0.R3 | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--alu-sr-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-alu-sr-1.0.1.signed.bin + > ./ncs-6.0-alu-sr-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-alu-sr-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-alu-sr-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `alu-sr-.`: + + ``` + > tar xfz ncs-6.0-alu-sr-1.0.1.tar.gz + > ls -d */ + alu-sr-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package alu-sr-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package alu-sr-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-alu-sr-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package alu-sr-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/alu-sr-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-alu-sr-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-alu-sr-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install alu-sr-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-alu-sr-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-alu-sr-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id alu-sr-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-alu-sr-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings alu-sr logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings alu-sr logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.alusr \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings alu-sr logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings alu-sr logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + The following is an example of configuring some security settings and adding them to a service vprn. + First step: add CLI commands in the NED CLI: + + ``` + admin@ncs(config)# devices device alu7750-5 config + admin@ncs(config-config)# system + admin@ncs(config-system)# security + admin@ncs(config-security)# pki + admin@ncs(config-pki)# ca-profile "PkiNokiaCa" + admin@ncs(config-ca-profile-PkiNokiaCa)# auto-crl-update + admin@ncs(config-auto-crl-update)# shutdown + admin@ncs(config-auto-crl-update)# exit + admin@ncs(config-ca-profile-PkiNokiaCa)# exit + admin@ncs(config-pki)# exit + admin@ncs(config-security)# exit + admin@ncs(config-system)# exit + admin@ncs(config-config)# ipsec + admin@ncs(config-ipsec)# ike-policy 2 + admin@ncs(config-ike-policy-2)# auth-method cert-auth + admin@ncs(config-ike-policy-2)# dpd interval 10 max-retries 2 reply-only + admin@ncs(config-ike-policy-2)# ike-version 2 + admin@ncs(config-ike-policy-2)# ipsec-lifetime 172800 + admin@ncs(config-ike-policy-2)# ike-transform 1 + admin@ncs(config-ike-policy-2)# exit + admin@ncs(config-ipsec)# ike-policy 3 + admin@ncs(config-ike-policy-3)# auth-method cert-auth + admin@ncs(config-ike-policy-3)# dpd interval 10 max-retries 2 reply-only + admin@ncs(config-ike-policy-3)# ike-version 2 + admin@ncs(config-ike-policy-3)# ipsec-lifetime 172800 + admin@ncs(config-ike-policy-3)# ike-transform 1 + admin@ncs(config-ike-policy-3)# exit + admin@ncs(config-ipsec)# ike-transform 1 + admin@ncs(config-ike-transform-1)# isakmp-lifetime 172800 + admin@ncs(config-ike-transform-1)# exit + admin@ncs(config-ipsec)# ipsec-transform 2 + admin@ncs(config-ipsec-transform-2)# exit + admin@ncs(config-ipsec)# cert-profile cert-profile-1 + admin@ncs(cert-profile)# entry 1 + admin@ncs(config-entry-1)# cert SEGW-PKI.crt + admin@ncs(config-entry-1)# key SEGW-PKI.key + admin@ncs(config-entry-1)# exit + admin@ncs(cert-profile)# no shutdown + admin@ncs(cert-profile)# exit + admin@ncs(config-ipsec)# cert-profile cert-profile-2 + admin@ncs(cert-profile)# entry 1 + admin@ncs(config-entry-1)# cert SEGW-PKI.crt + admin@ncs(config-entry-1)# key SEGW-PKI.key + admin@ncs(config-entry-1)# exit + admin@ncs(cert-profile)# no shutdown + admin@ncs(cert-profile)# exit + admin@ncs(config-ipsec)# trust-anchor-profile Trust-A-Profile-1 + admin@ncs(trust-anchor-profile)# trust-anchor "PkiNokiaCa" + admin@ncs(trust-anchor-profile)# exit + admin@ncs(config-ipsec)# trust-anchor-profile Trust-A-Profile-2 + admin@ncs(trust-anchor-profile)# trust-anchor "PkiNokiaCa" + admin@ncs(trust-anchor-profile)# exit + admin@ncs(config-ipsec)# tunnel-template 1 + admin@ncs(config-tunnel-template-1)# sp-reverse-route + admin@ncs(config-tunnel-template-1)# replay-window 128 + admin@ncs(config-tunnel-template-1)# transform 2 + admin@ncs(config-tunnel-template-1)# exit + admin@ncs(config-ipsec)# tunnel-template 2 + admin@ncs(config-tunnel-template-2)# sp-reverse-route + admin@ncs(config-tunnel-template-2)# replay-window 128 + admin@ncs(config-tunnel-template-2)# transform 2 + admin@ncs(config-tunnel-template-2)# exit + admin@ncs(config-ipsec)# exit + admin@ncs(config-config)# isa + admin@ncs(config-isa)# tunnel-group 1 isa-scale-mode tunnel-limit-32k + admin@ncs(config-tunnel-group-1)# description SecGW + admin@ncs(config-tunnel-group-1)# multi-active + admin@ncs(config-tunnel-group-1)# reassembly 2000 + admin@ncs(config-tunnel-group-1)# exit + admin@ncs(config-isa)# service + admin@ncs(config-service)# vprn 1234 customer 1 name 1234 + admin@ncs(vprn)# interface test + admin@ncs(interface)# dynamic-tunnel-redundant-next-hop 1.1.1.1 + admin@ncs(interface)# sap tunnel-1.public:2 + admin@ncs(sap)# ipsec-gw Ipsecgw-1 + admin@ncs(config-ipsec-gw)# shutdown + admin@ncs(config-ipsec-gw)# cert + admin@ncs(config-cert)# cert-profile cert-profile-1 + admin@ncs(config-cert)# status-verify + admin@ncs(config-status-verify)# default-result good + admin@ncs(config-status-verify)# exit + admin@ncs(config-cert)# trust-anchor-profile Trust-A-Profile-2 + admin@ncs(config-cert)# exit + admin@ncs(config-ipsec-gw)# default-secure-service 2000 interface private-interface-Ipsecgw-1 + admin@ncs(config-ipsec-gw)# default-tunnel-template 2 + admin@ncs(config-ipsec-gw)# ike-policy 3 + admin@ncs(config-ipsec-gw)# local-gateway-address 1.1.1.2 + admin@ncs(config-ipsec-gw)# local-id type fqdn value 1234 + admin@ncs(config-ipsec-gw)# exit + admin@ncs(sap)# exit + admin@ncs(interface)# exit + admin@ncs(vprn)# exit + admin@ncs(config-service)# vprn 2000 customer 1 + admin@ncs(vprn)# interface loopback_DATA_MOBILE + admin@ncs(interface)# exit + admin@ncs(vprn)# vprn 3000 customer 1 + admin@ncs(vprn)# interface loopback_DATA_MOBILE + admin@ncs(interface)# exit + admin@ncs(vprn)# exit + admin@ncs(config-service)# exit + admin@ncs(config-config)# + ``` + + Checking the device native CLI output and then commit changes: + + ``` + admin@ncs(config-service)# commit dry-run outformat native + native { + device { + name alu7750-5 + data exit all + configure + ipsec + ike-policy 2 create + ike-version 2 + auth-method cert-auth + dpd interval 10 max-retries 2 reply-only + exit + ike-transform 1 create + isakmp-lifetime 172800 + exit + ike-policy 2 create + ike-transform 1 + ipsec-lifetime 172800 + exit + ike-policy 3 create + ike-version 2 + auth-method cert-auth + dpd interval 10 max-retries 2 reply-only + ike-transform 1 + ipsec-lifetime 172800 + exit + ipsec-transform 2 create + exit + cert-profile cert-profile-1 create + entry 1 create + cert SEGW-PKI.crt + key SEGW-PKI.key + exit + no shutdown + exit + cert-profile cert-profile-2 create + entry 1 create + cert SEGW-PKI.crt + key SEGW-PKI.key + exit + no shutdown + exit + trust-anchor-profile Trust-A-Profile-1 create + exit + exit + system + security + pki + ca-profile PkiNokiaCa create + auto-crl-update create + shutdown + exit + exit + exit + exit + exit + ipsec + trust-anchor-profile Trust-A-Profile-1 create + trust-anchor PkiNokiaCa + exit + trust-anchor-profile Trust-A-Profile-2 create + trust-anchor PkiNokiaCa + exit + tunnel-template 1 create + sp-reverse-route + replay-window 128 + transform 2 + exit + tunnel-template 2 create + sp-reverse-route + replay-window 128 + transform 2 + exit + exit + isa + tunnel-group 1 isa-scale-mode tunnel-limit-32k create + description "SecGW" + multi-active + reassembly 2000 + exit + exit + service + vprn 1234 customer 1 name 1234 create + interface test create + sap tunnel-1.public:2 create + ipsec-gw Ipsecgw-1 + cert + cert-profile cert-profile-1 + status-verify + default-result good + exit + trust-anchor-profile Trust-A-Profile-2 + exit + exit + exit + exit + exit + vprn 2000 customer 1 name 2000 create + interface loopback_DATA_MOBILE create + exit + exit + vprn 1234 customer 1 name 1234 create + interface test create + sap tunnel-1.public:2 create + ipsec-gw Ipsecgw-1 + default-secure-service 2000 interface private-interface-Ipsecgw-1 + default-tunnel-template 2 + ike-policy 3 + local-gateway-address 1.1.1.2 + local-id type fqdn value 1234 + shutdown + exit + exit + dynamic-tunnel-redundant-next-hop 1.1.1.1 + exit + exit + vprn 3000 customer 1 name 3000 create + interface loopback_DATA_MOBILE create + exit + exit + exit + exit all + } + } + admin@ncs(config-config)# commit + Commit complete. + ``` + + Checking if there are diffs between the device configuration and the NED CDB: + + ``` + admin@ncs(config-config)# compare-config + admin@ncs(config-config)# check-sync + result in-sync + admin@ncs(config-config)# + ``` + + In the above commands, compare-config result should be empty and the check-sync result should be in-sync. If there are different results, it is possible that some commands failed on the device (the device output may contain errors that are not caught by the NED) or the device added some dynamic data. Trace analysis is needed in this case to understand what went wrong. + If the NED missed a device error that it was supposed to be thrown by the NED, please check chapter 7 from this document and README-ned-settings.md chapter 13.1 on how to address this issue. + + The commited changes can also be rolled back to the original device state, before the commit: + + ``` + admin@ncs(config-config)# top rollback config + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name alu7750-5 + data exit all + configure + ipsec + no ike-policy 2 + exit + service + vprn 1234 customer 1 name 1234 create + interface test create + no dynamic-tunnel-redundant-next-hop + sap tunnel-1.public:2 create + ipsec-gw Ipsecgw-1 + cert + no cert-profile + status-verify + no default-result + exit + no trust-anchor-profile + exit + no default-secure-service + no default-tunnel-template + no ike-policy + no local-gateway-address + no local-id + shutdown + exit + no ipsec-gw + exit + sap tunnel-1.public:2 shutdown + no sap tunnel-1.public:2 + exit + exit + exit + ipsec + no ike-policy 3 + no ike-transform 1 + no tunnel-template 1 + no tunnel-template 2 + no ipsec-transform 2 + cert-profile cert-profile-1 shutdown + no cert-profile cert-profile-1 + cert-profile cert-profile-2 shutdown + no cert-profile cert-profile-2 + no trust-anchor-profile Trust-A-Profile-1 + no trust-anchor-profile Trust-A-Profile-2 + exit + isa + tunnel-group 1 isa-scale-mode tunnel-limit-32k create + no description + no multi-active + no reassembly + exit + no tunnel-group 1 + exit + service + vprn 1234 customer 1 name 1234 create + interface test shutdown + no interface test + exit + vprn 1234 shutdown + no vprn 1234 + vprn 2000 customer 1 name 2000 create + interface loopback_DATA_MOBILE shutdown + no interface loopback_DATA_MOBILE + exit + vprn 2000 shutdown + no vprn 2000 + vprn 3000 customer 1 name 3000 create + interface loopback_DATA_MOBILE shutdown + no interface loopback_DATA_MOBILE + exit + vprn 3000 shutdown + no vprn 3000 + exit + system + security + pki + ca-profile PkiNokiaCa shutdown + no ca-profile PkiNokiaCa + exit + exit + exit + exit all + } + } + admin@ncs(config-config)# commit + Commit complete. + admin@ncs(config-config)# compare-config + admin@ncs(config-config)# + ``` + + +# 5. Built in live-status actions +--------------------------------- + + The NED supports the following live-status exec commands: + - any: Execute any command on device + - ping: Send echo message + - debug: Execute debug commands + - show: Execute show commands + - admin: Execute admin commands + - any encryption re-encrypt obfuscated: this is used to force the secrets operational CDB update (please check chapter 9) + + +# 6. Built in live-status show +------------------------------ + + ALU-SR NED supports a bunch of live-status 'show' TTL-based commands. Here is a list of supported elements: + + ``` + admin@ncs# show devices device alu7750-5 live-status + Possible completions: + card Display Card information + chassis Display chassis information + eth-cfm + lag + modules-state + ports Display all available ports (on all slots) + resource-usage + router Display router instance information + service Display services related information + sfm Display Switch Fabric Module (sfm) information + slot Display MDA information + system Display system params + + ``` + + Example of a live-status call: + + ``` + admin@ncs# show devices device alu7750-5 live-status slot + live-status slot 1 + mda 1 + ports-maximum 20 + ports-equipped 20 + last-boot 2023-04-27T08:18:26-00:00 + oper-state up + admin-state up + software-version "(Not Specified)" + provisioned-type m20-v + admin@ncs# + + ``` + + +# 7. Limitations +---------------- + + Device errors: since ALU-SR is not a CISCO device, the NED does not have a complete list of device errors to be thrown. + + This may lead to device errors that are missed by the NED, usually generating a compare-config issue. + However, it is possible to define a custom list of device errors to be thrown by the NED. + Please check the README-ned-settings.md for ned-settings alu-sr transaction config-abort-warning + + YANG model: the NED behavior is not 1 to 1 with the device behavior because of the yang limitations. + For this reason, there are several work-around in the NED to achieve a behavior similar to the device. + For example, a leaf 'encap-type' that allows values like 'dot1q' but also allows a value like 'no encap-type' (no encap-type shown on the device), this leaf may be modeled as 'encap-type no' to also support the device 'no encap-type'. + + Redeployment: a significant number of device operations requires redeployment of configuration. + For instance, updates to a specific node may need to shutdown an external node first, do the update and then restore the external node state. + While redeployments are common and vastly supported in ALU-SR NED, the redeployment is always done with the current NED data. + For instance, the NED cannot redeploy data if the targeted data for redeployment is also changed in the current transaction. + + Custom trans-id methods: using ned-settings (please check README-ned-settings.md), it is possible to change the way alu-sr NED computes trans-id. + Popular choices for trans-id computation are config-hash or config-data, which basically pulls the entire device configuration in order to compute trans-id hash. + These two methods are most reliable, since they catch any changes on the device, however, most of the time, device configurations are significantly large, which introduces significant get and processing time. + To drastically reduce the trans-id computation time, alu-sr provides custom trans-id methods, but they have certain limitations: + - rollback-timestamp: trans-id computation based on the device rollback timestamp (device rollback feature must be on). This method is unreliable because the rollback timestamp does not change when the device config is changed, only when the device is rolled back. + - last-modified-timestamp: cumputes trans-id based on the device last modified timestamp. This method is reliable to detect out-of-band changes, as long as the device configuration is not saved out-of-band (eg execute 'admin save' on the device). When saving the device configuration out-of-band (eg 'admin save'), this timestamp is invalidated by the device. + - last-saved-timestamp: this method uses the last saved device timestamp. This method is reliable as long as the device changes are saved every time, either from the NED or from the device. Note: out-of-band changes of the device that are not saved will not be detected by this method + - last-modified-and-last-saved-timestamp: this method uses both last modified and last saved timestamps. This method is reliabe to detect both out-of-band modifications or configuration save on the device. Limitation for this method is that the NED will be in out-of-sync state when the device configuration is not modified but it is saved (eg when 'admin save' is used out-of-band, the NED will be in out-of-sync state, even though the device configuration was not actually modified) + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings alu-sr logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings alu-sr logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` + +# 11. NED Secrets - Securing your Secrets +----------------------------------------- + + It is best practice to avoid storing your secrets (e.g. passwords and + shared keys) in plain-text, either on NSO or on the device. In NSO we + support multiple encrypted datatypes that are encrypted using a local + key, similarly many devices such as ALU-SR supports automatically + encrypting all passwords stored on the device. + + Naturally, for security reasons, NSO in general has no way of + encrypting/decrypting passwords with the secret key on the + device. This means that if nothing is done about this we will + become out of sync once we write secrets to the device. + + In order to avoid becoming out of sync the NED reads back these elements + immediately after set and stores the encrypted value(s) in a special + `secrets` table in oper data. Later on, when config is read from the + device, the NED replaces all cached encrypted values with their plaintext + values; effectively avoiding all config diffs in this area. If the values + are changed on the device, the new encrypted value will not match the + cached pair and no replacement will take place. This is desired, since out + of band changes should be detected. + + This handles the device-side encryption, but passwords are still unencrypted + in NSO. To deal with this we support using NSO-encrypted strings instead of + plaintext passwords in the NSO data model. + + --- Handling auto-encryption + + Let us say that we have password-encryption on and we want to write a new + user to our device: + + system + security + user admin + access console + console + member administrative + exit + password + + this will be automatically encrypted by the device + + *A:VSR-7750>config# system security user "admin" + *A:VSR-7750>config>system>security>user# info + ---------------------------------------------- + password "$2y$10$fBSDYG2MHpdpCTDQhq7BE.ojwFR5z10g61PUqWaXb52GXg0Ge8d8W" + access console + console + member "administrative" + exit + ---------------------------------------------- + + But the secrets management will store this new encrypted value in our `secrets` table: + + admin@ncs# show devices device dev-1 ned-settings secrets + ID ENCRYPTED REGEX + --------------------------------------------------------------------------------------------------------------- + /system/security/user_admin_/password/id $2y$10$fBSDYG2MHpdpCTDQhq7BE.ojwFR5z10g61PUqWaXb52GXg0Ge8d8W - + + which means that compare-config or sync-from will not show any + changes and will not result in any updates to CDB". In fact, we can + still see the unencrypted value in the device tree: + + admin@ncs(config-config)# show full sys sec user + devices device dev-1 + config + system + security + user admin + access console + console + member administrative + ! + password + + --- Increasing security with NSO-side encryption + + We have two alternatives, either we can manually encrypt our values using + one of the NSO-encrypted types (e.g `aes-256-cfb-128-encrypted-string`) and + set them to the tree, or we can recompile the NED to always encrypt secrets. + + --- Setting encrypted value + + Let us say we know that the NSO-encrypted string + `$2y$10$7ova9fF/bRe9B9GUtjVpA.w5mfeXJXRHyV0KsSfg4XWE9j3Fcq3Qi`, we + can then set it in the device tree as normal + + admin@ncs(config-config)# system security user admin password $2y$10$7ova9fF/bRe9B9GUtjVpA.w5mfeXJXRHyV0KsSfg4XWE9j3Fcq3Qi + admin@ncs(config-config)# commit + + when commiting this value it will be decrypted and the plaintext will be written to the device. + Unlike the previous example the plaintext is not visible in the device tree: + + admin@ncs(config-config)# show full sys sec user + devices device dev-1 + config + system + security + user admin + access console + console + member administrative + ! + password $2y$10$7ova9fF/bRe9B9GUtjVpA.w5mfeXJXRHyV0KsSfg4XWE9j3Fcq3Qi + + On the device side this plaintext value is of course encrypted + with the device key, and just as before we store it in our + `secrets` table: + + admin@ncs# show devices device dev-1 ned-settings secrets + ID ENCRYPTED REGEX + --------------------------------------------------------------------------------------------------------------- + /system/security/user_admin_/password/id $2y$10$fBSDYG2MHpdpCTDQhq7BE.ojwFR5z10g61PUqWaXb52GXg0Ge8d8W - + + We can see that this corresponds to the value set on the device. + + --- Auto-encrypting passwords in NSO + + To avoid having to pre-encrypt your passwords you can rebuild your NED in your OS + command shell specifying an encrypted type for secrets using a command like: + + yourhost:~/ned-folder$ NEDCOM_SECRET_TYPE="tailf:aes-cfb-128-encrypted-string" make -C src/ clean all + + Or by adding the line `NED_EXTRA_BUILDFLAGS ?= NEDCOM_SECRET_TYPE=tailf:aes-cfb-128-encrypted-string` + in top of the `Makefile` located in /src directory. + + Doing this means that even if the input to a password is a plaintext string, NSO will always + encrypt it, and you will never see plain text secrets in the device tree. + + If we reload our example with the new NED all of the secrets are now encrypted: + + admin@ncs(config-config)# show full sys sec user + devices device dev-1 + config + system + security + user admin + access console + console + member administrative + ! + password $2y$10$7ova9fF/bRe9B9GUtjVpA.w5mfeXJXRHyV0KsSfg4XWE9j3Fcq3Qi diff --git a/arista-dcs/README-ned-settings.md b/arista-dcs/README-ned-settings.md new file mode 100644 index 00000000..22379af6 --- /dev/null +++ b/arista-dcs/README-ned-settings.md @@ -0,0 +1,424 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/arista-dcs/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/arista-dcs/ + device + /ncs:/device/devices/device:/ned-settings/arista-dcs/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings arista-dcs + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings arista-dcs + 2. proxy + 3. connection + 4. logger + 5. live-status + 6. developer + 6.1. simulate-show + 7. read + 7.1. show-running-all-match-pattern + 8. write + 8.1. inject-command + ``` + + +# 1. ned-settings arista-dcs +---------------------------- + + arista-dcs ned-settings. + + + - extended-parser (default auto) + + Make the NED enable extensions to ease the task of the NSO CLI command parser. A common + problem with this parser is that it can easily get lost when trying to parse configuration not + supported by the YANG model. In particular it is very sensitive for unsupported config that + generates a mode switch. + + auto - Uses turbo-mode when available, will use fastest availablemethod to load + data to NSO. If NSO doesn't support data-loading from CLI NED, robust-mode + is used. + + robust-mode - The configuration dump is run through a pre-parser which is cleaning it from + all elements currently not supported in the YANG model (default). + + turbo-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO using a + Maapi SetValues() call. + + turbo-xml-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO in XML + format. + + +# 2. ned-settings arista-dcs proxy +---------------------------------- + + Configure NED to access device via a proxy. + + + - proxy remote-connection + + Connection type between proxy and device. + + ssh - ssh. + + telnet - telnet. + + serial - serial. + + + - proxy remote-address + + Address of host behind the proxy. + + + - proxy remote-port + + Port of host behind the proxy. + + + - proxy proxy-prompt + + Prompt pattern on the proxy host. + + + - proxy remote-prompt + + Prompt pattern on the remote (proxy) host. + + + - proxy remote-name + + User name on the device behind the proxy. + + + - proxy remote-password + + Password on the device behind the proxy. + + +# 3. ned-settings arista-dcs connection +--------------------------------------- + + Configure settings specific to the connection between NED and device. + + + - connection number-of-retries (default 0) + + Configure max number of retries the NED will try to connect to the device before giving up. + Default 0. + + + - connection time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + + - connection connector + + Change the default connector, e.g. 'ned-connector-default.json'. + + + - connection ssh client + + Configure the SSH client to use. Relevant only when using the NED with NSO 5.6 or later. + + ganymed - The legacy SSH client. Used on all older versions of NSO. + + sshj - The new SSH client with support for the latest crypto features. This is the default + when using the NED on NSO 5.6 or later. + + + - connection ssh host-key known-hosts-file + + Path to openssh formatted 'known_hosts' file containing valid host keys. + + + - connection ssh host-key public-key-file + + Path to openssh formatted public (.pub) host key file. + + + - connection ssh auth-key private-key-file + + Path to openssh formatted private key file. + + +# 4. ned-settings arista-dcs logger +----------------------------------- + + Settings for controlling logs generated. + + + - logger level (default info) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - logger java (default true) + + Toggle logs to be added to ncs-java-vm.log. + + +# 5. ned-settings arista-dcs live-status +---------------------------------------- + + Configure NED settings related to live-status. + + + - live-status time-to-live (default 50) + + Define time-to-live for data fetched from the device via live-status.(default 50). + + + - live-status auto-prompts + + See chapter 5 in README.md for information on this ned-setting. + + - id + + List id, any string. + + - question + + Device question, regular expression. + + - answer + + Answer to device question. + + +# 6. ned-settings arista-dcs developer +-------------------------------------- + + Contains settings used for debugging (intended for NED developers). + + + - developer progress-verbosity (default debug) + + Maximum NED verbosity level which will get written in devel.log file. + + disabled - disabled. + + normal - normal. + + verbose - verbose. + + very-verbose - very-verbose. + + debug - debug. + + + - developer model + + Simulate a model number. + + + - developer version + + Simulate a version number. + + + - developer trace-connection (default false) + + Enable developer connection tracing. WARNING: may choke NSO with large printouts. + + + - developer platform model + + Override device model name/number. + + + - developer platform name + + Override device name. + + + - developer platform version + + Override device version. + + +## 6.1. ned-settings arista-dcs developer simulate-show +------------------------------------------------------- + + Used with live-status to inject simulated output for a show command. + + - developer simulate-show + + - cmd + + Full show command, e.g. 'show interfaces'. + + - file + + Path to file containing output of simulated show command. + + +# 7. ned-settings arista-dcs read +--------------------------------- + + Settings used when reading from device. + + + - read show-running-method (default show running-config) + + Normally the NED uses "show running-config" to show configuration. + This can be changed using this ned-setting, for example: + + devices device dev-1 ned-settings arista-dcs read show-running-method + "show running-config all | include (interface Ethernet)" + + Note: Using custom show running-config commands could lead to NED + unexpected behavior and performance issues, therefore it is strongly + recommended to be careful with it. + + +## 7.1. ned-settings arista-dcs read show-running-all-match-pattern +------------------------------------------------------------------- + + The list of the patterns included in the "show running-config all" command, for example: + + devices device arista ned-settings arista-dcs read show-running-all-match-pattern "^logging trap" + devices device arista ned-settings arista-dcs read show-running-all-match-pattern "^aaa authorization config-commands" + + - read show-running-all-match-pattern + + - regexp + + Regular Expression. + + +# 8. ned-settings arista-dcs write +---------------------------------- + + Settings used when writing to device. + + + - write interface-mac-address-learning-wait-before-configure (default 0) + + This setting allows for configuring a small delay before setting of an interface 'mac address + learning disabled'. Some use cases require a small delay before this configuration being + available. + + + - write memory-setting (default on-commit) + + Configure how and when an applied config is saved to persistent memory on the device. + + on-commit - Save configuration immediately after the config has been successfully applied on + the device. If an error occurs when saving the whole running config will be + rolled back. + + on-persist - Save configuration during the NED persist handler. Called after the config has + been successfully applied and commited If an error occurs when saving an alarm + will be triggered. No rollback of the running config is done. + + disabled - Disable saving the applied config to persistent memory. + + +## 8.1. ned-settings arista-dcs write inject-command +---------------------------------------------------- + + - write inject-command + + The arista-dcs write inject-command ned-setting can be used to inject + command line(s) in a transaction. This can be needed, for example, + when deleting crypto config which requires a clear command to be + run before delete. + + The ned-settings is configured with: + + id + User defined name for this ned-setting used to identify the list entry + + config-line + The config line(s) where command should be injected (DOTALL regexp) + + command + The command (or config) to inject after|before config-line. + Prefix with 'do ' if you want to run exec command in config mode. + Prefix with 'exec ' if you want to run exec command in exec mode. + + 'where', eight values are supported: + before-each + inject command before each matching + before-first + inject command before first matching + after-each + inject command after each matching + after-last + inject command after last matching + before-topmode + inject command before regex topmode + after-topmode + inject command after regex topmode + first + inject command first if regex matches or is unset + last + inject command last if regex matches or is unset + + An example (of a previously hard coded inject case): + + devices global-settings ned-settings arista-dcs write inject-command C1 config-line + "no crypto ikev2 keyring \\S+" command "do clear crypto session" before-first + devices global-settings ned-settings arista-dcs write inject-command C2 config-line + "no crypto ikev2 keyring \\S+" command "do clear crypto ikev2 sa fast" before-first + + The above inject command configs will cause a delete of ikev2 keyring to + look like this: + + do clear crypto session + do clear crypto ikev2 sa fast + no crypto ikev2 keyring XXX + + $i (where i is value from 1 to 9) can also be used to inject + matches values from the config line. For example: + + devices global-settings ned-settings arista-dcs write inject-command C2 config-line + "no interface Tunnel(\\d+)" command "do clear dmvpn session interface Tunnel $1 static" before-first + + with a deletion of interface Tunnel100 results in: + + !do clear dmvpn session interface Tunnel 100 static + no interface Tunnel100 + + Hence, $1 is replaced with the first group value from the config line, + which is (\\d+). + + diff --git a/arista-dcs/README.md b/arista-dcs/README.md new file mode 100644 index 00000000..24af03c6 --- /dev/null +++ b/arista-dcs/README.md @@ -0,0 +1,1058 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + 11. NED Secrets - Securing your Secrets + ``` + + +# 1. General +------------ + + This document describes the arista-dcs NED. + + Arista-dcs NED is built for Arista devices running DCS, EOS and vEOS. + + The NED connects to the device CLI using either SSH or + Telnet. + + Configuration is done by sending native CLI commands in a + transaction to the device through the communication channel. If a + single command fails, the whole transaction is aborted and reverted. + + If you suspect a bug in the NED, please see chapter 8. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | - | + | | | | + | check-sync | yes | - | + | | | | + | partial-sync-from | yes | - | + | | | | + | live-status actions | yes | - | + | | | | + | live-status show | yes | - | + | | | | + | load-native-config | yes | - | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Arista | ['4.1x', | DCS | - | + | | '4.2x', '4.3x'] | | | + | | | | | + | Arista | ['4.1x', | EOS | - | + | | '4.2x', '4.3x'] | | | + | | | | | + | Arista | ['4.1x', | vEOS | - | + | | '4.2x', '4.3x'] | | | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--arista-dcs-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-arista-dcs-1.0.1.signed.bin + > ./ncs-6.0-arista-dcs-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-arista-dcs-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-arista-dcs-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `arista-dcs-.`: + + ``` + > tar xfz ncs-6.0-arista-dcs-1.0.1.tar.gz + > ls -d */ + arista-dcs-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package arista-dcs-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package arista-dcs-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-arista-dcs-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package arista-dcs-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/arista-dcs-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-arista-dcs-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-arista-dcs-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install arista-dcs-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-arista-dcs-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-arista-dcs-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id arista-dcs-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-arista-dcs-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings arista-dcs logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings arista-dcs logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.dcs \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings arista-dcs logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings arista-dcs logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + **SSHJ DEBUG LOGGING** + For issues related to the ssh connection it is often useful to enable full logging in the SSHJ ssh client. + This will make SSHJ print additional log entries in `$NSO_RUNDIR/logs/ncs-java-vm.log`: + +``` +admin@ncs(config)# java-vm java-logging logger net.schmizz.sshj level level-all +admin@ncs(config)# commit +``` + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + For instance, create a Loopback interface that is down: + + ``` + admin@ncs(config)# devices device dev-1 config + admin@ncs(config-config)# interface Loopback 1 + admin@ncs(config-if)# ip address 128.0.0.1/8 + admin@ncs(config-if)# shutdown + ``` + + See what you are about to commit: + + ``` + admin@ncs(config-if)# commit dry-run outformat native + native { + device { + name dev-1 + data interface Loopback1 + ip address 128.0.0.1/8 + shutdown + exit + } + } + ``` + + Commit new configuration in a transaction: + + ``` + admin@ncs(config-if)# commit + Commit complete. + ``` + + Verify that NCS is in-sync with the device: + + ``` + admin@ncs(config-if)# devices device dev-1 check-sync + result in-sync + ``` + + Compare configuration between device and NCS: + + ``` + admin@ncs(config-if)# devices device dev-1 compare-config + admin@ncs(config-if)# + ``` + + Note: if no diff is shown, supported config is the same in + NCS as on the device. + + +# 5. Built in live-status actions +--------------------------------- + + The NED has support for all operational Arista DCS exec commands + by use of the 'devices device dev-1 live-status exec any' action. + + For example: + + ``` + admin@ncs(config)# devices device dev-1 live-status exec any "show clock" + result + Wed May 10 06:45:44 2023 + Timezone: UTC + Clock source: local + + admin@ncs(config)# + ``` + + Generally the command output parsing halts when the NED detects + an operational or config prompt, however sometimes the command + requests additional input, 'answer(s)' to questions. + + To respond to device question(s) there are 2 different methods, + checked in the listed order below: + + [1] the ned-settings arista-dcs live-status auto-prompts list + [2] the command line args "| prompts" option + + IMPORTANT: [2] can be used to override an answer in auto-prompts. + + Read on for details on each method: + + [1] ned-settings arista-dcs live-status auto-prompts list + + The auto-prompts list is used to pass answers to questions, to + exit parsing, reset timeout or ignore output which triggered the + the built-in question handling. Each list entry contains a question + (regex format) and an optional answer (text or built-in keyword). + + + Here are some examples of auto-prompts ned-settings: + + ``` + devices global-settings ned-settings arista-dcs live-status auto-prompts Q1 question "System configuration has been modified" answer "no" + devices global-settings ned-settings arista-dcs live-status auto-prompts Q2 question "Do you really want to remove these keys" answer "yes" + devices global-settings ned-settings arista-dcs live-status auto-prompts Q3 question "Press RETURN to continue" answer ENTER + ``` + + NOTE: Due to backwards compatibility, ned-setting auto-prompts + questions get ".*" appended to their regex unless ending with + "$". + + [2] "| prompts" + + "| prompts" is passed in the command args string and is used to + submit answer(s) to the device without a matching question pattern. + IMPORTANT: It can also be used to override answer(s) configured in + auto-prompts list, unless the auto-prompts contains or + , which are always handled first. + + One or more answers can be submitted following this syntax: + + | prompts .. [answer N] + + For example: + + devices device dev-1 live-status exec any "reload | prompts no yes" + + The following output of the device triggers the NED to look for the + answer in | prompts arguments: + + ":\\s*$" + "\\][\\?]?\\s*$" + + In other words, the above two patterns (questions) have a built-in + for an answer. + + Additional patterns triggering | prompts may be configured by use + of auto-lists and setting the answer to . This will force + the user to specify the answer in | prompts. + + The or IGNORE keywords can be used to ignore device output + matching the above and continue parsing. If all output should be + ignored, i.e. for a show command, '| noprompts' should be used. + + Some final notes on the 'answer' leaf: + + - "ENTER" or means a carriage return + line feed is sent. + + - "IGNORE", "" or unset means the prompt was not a + question, the device output is ignored and parsing continues. + + - A single letter answer is sent without carriage return + line, + i.e. "N" will be sent as N only, with no return. If you want a + return, set "NO" as the answer instead. + + +# 6. Built in live-status show +------------------------------ + + Arista-dcs NED currently supports below live-status commands: + + ``` + admin@ncs# show devices device live-status ? + Description: Status data fetched from the device + Possible completions: + interfaces mac mac-address-table port-channel | + admin@ncs# + ``` + + For instance, below is shown the output of "show interfaces transceiver" command: + + ``` + admin@ncs# show devices device dev-1 live-status interfaces transceiver + OPTICAL OPTICAL + BIAS TX RX LAST + NAME TEMP VOLTAGE CURRENT POWER POWER UPDATE + --------------------------------------------------------------- + Ethernet1 - - - - - - + Ethernet2 - - - - - - + Ethernet3 - - - - - - + Ethernet4 - - - - - - + Ethernet5 - - - - - - + Ethernet6 - - - - - - + Ethernet7 - - - - - - + Ethernet8 - - - - - - + Loopback1 - - - - - - + Management1 - - - - - - + + admin@ncs# + ``` + + +# 7. Limitations +---------------- + + NONE + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings arista-dcs logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings arista-dcs logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` + +# 11. NED Secrets - Securing your Secrets +----------------------------------------- + +Naturally, for security reasons, NSO in general has no way of +encrypting/decrypting passwords with the secret key on the +device. This means that if nothing is done about this we will +become out of sync once we write secrets to the device. + +In order to avoid becoming out of sync the NED reads back these elements +immediately after set and stores the encrypted value(s) in a special +'secrets' table in oper data. Later on, when config is read from the +device, the NED replaces all cached encrypted values with their plaintext +values; effectively avoiding all config diffs in this area. If the values +are changed on the device, the new encrypted value will not match the +cached pair and no replacement will take place. This is desired, since out +of band changes should be detected. + +This handles the device-side encryption, but passwords are still unencrypted +in NSO. To deal with this we support using NSO-encrypted strings instead of +plaintext passwords in the NSO data model. + +--- Handling auto-encryption + +Let us say that we have password-encryption on and we want to write a new +user to our device: + + ``` + admin@ncs(config)# devices device dev-1 config username newuser secret 0 magic + admin@ncs(config-config)# commit + ``` + +this will be automatically encrypted by the device + + ``` + Router#show running-config | section usernaname + username newuser secret sha512 $6$NkESssLkb8SoEYcj$GnapsHSptZE9PYxI2A28VY.j/BrwwOHrpC.8zEpX1LfxiwTIxkpj6gDIaNsis7VVHHWfSd2ipa3WxTmQBMCxq/ + ``` + +But the secrets management will store this new encrypted value in our 'secrets' table: + + ``` + admin@ncs# show devices device dev-1 ned-settings secrets + ID ENCRYPTED + --------------------------------------------------------------------------------------------------------------------------------------------------------- + dcs:username(newuser)/secret/secret sha512 $6$NkESssLkb8SoEYcj$GnapsHSptZE9PYxI2A28VY.j/BrwwOHrpC.8zEpX1LfxiwTIxkpj6gDIaNsis7VVHHWfSd2ipa3WxTmQBMCxq/ + ``` + +which means that compare-config or sync-from will not show any +changes and will not result in any updates to CDB". In fact, we can +still see the unencrypted value in the device tree: + + ``` + admin@ncs# show running-config devices device dev-1 config username + devices device dev-1 + config + username admin role network-admin secret sha512 $6$1AFikEpbjqGE528Y$Wfiod2mV2MOwdu5OD6HLTlbu/MPsn/AG4F0b1DwPXApjScvH481w.swLkqnMOBC0Fg0uzV94hXCtAtd5bLnlf0 + username newuser secret 0 magic + ! + ! + ``` + +--- Increasing security with NSO-side encryption + +We have two alternatives, either we can manually encrypt our values using +one of the NSO-encrypted types (e.g 'aes-256-cfb-128-encrypted-string') and +set them to the tree, or we can recompile the NED to always encrypt secrets. + +--- Setting encrypted value + +Let us say we know that the NSO-encrypted string +'$9$T963R76+wgaQuZCtcGC/Nreo75FigP+znmOln8XDFK0=' ('admin'), we +can then set it in the device tree as normal + + ``` + admin@ncs(config)# devices device dev-1 config username newuser2 secret 0 $9$T963R76+wgaQuZCtcGC/Nreo75FigP+znmOln8XDFK0= + admin@ncs(config-config)# commit + ``` + +when commiting this value it will be decrypted and the plaintext will be written to the device. +Unlike the previous example the plaintext is not visible in the device tree: + + ``` + admin@ncs# show running-config devices device dev-1 config username + devices device dev-1 + config + username admin role network-admin secret sha512 $6$1AFikEpbjqGE528Y$Wfiod2mV2MOwdu5OD6HLTlbu/MPsn/AG4F0b1DwPXApjScvH481w.swLkqnMOBC0Fg0uzV94hXCtAtd5bLnlf0 + username newuser secret 0 magic + username newuser2 secret 0 $9$T963R76+wgaQuZCtcGC/Nreo75FigP+znmOln8XDFK0= + ! + ! + ``` + +On the device side this plaintext value is of course encrypted +with the device key, and just as before we store it in our +'secrets' table: + + ``` + admin@ncs# show devices device dev-1 ned-settings secrets + ID ENCRYPTED + --------------------------------------------------------------------------------------------------------------------------------------------------------- + dcs:username(newuser)/secret/secret sha512 $6$NkESssLkb8SoEYcj$GnapsHSptZE9PYxI2A28VY.j/BrwwOHrpC.8zEpX1LfxiwTIxkpj6gDIaNsis7VVHHWfSd2ipa3WxTmQBMCxq/ + dcs:username(newuser2)/secret/secret sha512 $6$STbF4/rMe4ojOCys$OgZMw0ItG6/Drg7GJQRY4istytEf5fIvTep3L4K3eeePY3ST80cs3Ui5J6upAMoX4y8bJlWfBDsZiN9Qoz2yx0 + ``` + +We can see that this corresponds to the value set on the device: + + ``` + Router#show running-config | section username + username admin role network-admin secret sha512 $6$1AFikEpbjqGE528Y$Wfiod2mV2MOwdu5OD6HLTlbu/MPsn/AG4F0b1DwPXApjScvH481w.swLkqnMOBC0Fg0uzV94hXCtAtd5bLnlf0 + username newuser secret sha512 $6$NkESssLkb8SoEYcj$GnapsHSptZE9PYxI2A28VY.j/BrwwOHrpC.8zEpX1LfxiwTIxkpj6gDIaNsis7VVHHWfSd2ipa3WxTmQBMCxq/ + username newuser2 secret sha512 $6$STbF4/rMe4ojOCys$OgZMw0ItG6/Drg7GJQRY4istytEf5fIvTep3L4K3eeePY3ST80cs3Ui5J6upAMoX4y8bJlWfBDsZiN9Qoz2yx0 + ``` + +Note that we do not have entries for 'admin', because these +values were set directly on NSO and not on the device, we +do not have a corresponding plaintext value and they are handled +entirely as encrypted values. + +--- Auto-encrypting passwords in NSO + +To avoid having to pre-encrypt your passwords you can rebuild your NED with an encrypted type for secrets using +a command like 'NEDCOM_SECRET_TYPE="tailf:aes-cfb-128-encrypted-string" make -C +src/'. Doing this means that even if the input to a password is a plaintext string, +NSO will always encrypt it, and you will never see plain text secrets in the device tree. + +If we reload our example with the new NED all of the secrets are now encrypted: + + ``` + admin@ncs# show running-config devices device dev-1 config username + devices device dev-1 + config + username admin role network-admin secret sha512 "$8$sIonje7gXru0LPL/CSyL0o08Qs1p65PM/SVQnOjQkLfXiZ2JghvDpB3u+OGp2uFqOE9ci3yQ\nqQF9SuZBbg3bzfbtDaSjArmHkj2AHF8lesGQfm5eSXllIT7k5P4a2KX+/EsezTFuy5QsLqaG\nvckAUPngjMJcpkf0X+997Yy8etY=" + username newuser secret 0 $8$j47TfNk/Ml3TFrVvBb14gxe9pKf8PASb78N07fzaflI= + username newuser2 secret 0 $8$g7CX0j+pfEiFDktw9+01XVTe/8R0dGb4MXkwJcQsvhg= + ! + ! + ``` + +and if we create yet another user we get the desired result: + + ``` + admin@ncs(config-config)# username newuser3 secret 0 MY-AUTO-PASSWD + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name dev-1 + data username newuser3 secret 0 $8$mtTsRpjsFY0PxN/oertlZRRuRvwoI4lddEOZak4lb90= + } + } + admin@ncs(config-config)# commit + Commit complete. + admin@ncs(config-config)# end + admin@ncs# show running-config devices device dev-1 config username newuser3 + devices device dev-1 + config + username newuser3 secret 0 $8$mtTsRpjsFY0PxN/oertlZRRuRvwoI4lddEOZak4lb90= + ! + ! + + admin@ncs# show devices device dev-1 ned-settings secrets + ID ENCRYPTED + --------------------------------------------------------------------------------------------------------------------------------------------------------- + dcs:username(newuser)/secret/secret sha512 $6$NkESssLkb8SoEYcj$GnapsHSptZE9PYxI2A28VY.j/BrwwOHrpC.8zEpX1LfxiwTIxkpj6gDIaNsis7VVHHWfSd2ipa3WxTmQBMCxq/ + dcs:username(newuser2)/secret/secret sha512 $6$STbF4/rMe4ojOCys$OgZMw0ItG6/Drg7GJQRY4istytEf5fIvTep3L4K3eeePY3ST80cs3Ui5J6upAMoX4y8bJlWfBDsZiN9Qoz2yx0 + dcs:username(newuser3)/secret/secret sha512 $6$o2sRu/2oMvhqX4fs$u0CuFpYiloWlBkjd89Gm0ud6tWXVuqH0LxTQ5mvpChfA1k0JU7XNSzfiOMNp0XZelga31KAM8a0G1HD9QgnIm/ + ``` diff --git a/arris-cmts/README-ned-settings.md b/arris-cmts/README-ned-settings.md new file mode 100644 index 00000000..705afb84 --- /dev/null +++ b/arris-cmts/README-ned-settings.md @@ -0,0 +1,340 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/arris-cmts/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/arris-cmts/ + device + /ncs:/device/devices/device:/ned-settings/arris-cmts/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings arris-cmts + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings arris-cmts + 2. connection + 3. deprecated + 4. transaction + 5. development_settings + 6. proxy + 7. logger + 8. developer + 9. proxy-settings + 10. ignore-error-messages + ``` + + +# 1. ned-settings arris-cmts +---------------------------- + + Configure settings specific to the connection between NED and device. + + + - arris-cmts extended-parser (default auto) + + Make the NED handle CLI parsing (i.e. transform the running-config from the device to the + model based config tree). + + auto - Uses turbo-mode when available, will use fastest availablemethod to load + data to NSO. If NSO doesn't support data-loading from CLI NED, robust-mode + is used. + + turbo-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO using maapi + setvalues call. + + turbo-xml-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO in XML + format. + + robust-mode - Makes the NED filter the configuration so that unmodeled content is removed + before being passed to the NSO CLI-engine. This protects against + configuration ending up at the wrong level when NSO CLI parser fallbacks + (which potentially can cause following config to be skipped). + + + - arris-cmts persist (default true) + + this option will disable the 'copy running config to start up config' command that's issued + after commit. + + + - arris-cmts confFromFileEnable (default false) + + choose if a config file should be used instead of a real device. + + + - arris-cmts confFromFileName + + Specify a file to be used for sync-from, instead of a real device. + + +# 2. ned-settings arris-cmts connection +--------------------------------------- + + Configure settings specific to the connection between NED and device. + + + - connection number-of-retries (default 0) + + Configure max number of retries the NED will try to connect to the device before giving up. + Default 0. + + + - connection time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + + - connection connector + + Change the default connector, e.g. 'ned-connector-default.json'. + + + - connection ssh client (default ganymed) + + Configure the SSH client to use. Relevant only when using the NED with NSO 5.6 or later. + + ganymed - The legacy SSH client. Used on all older versions of NSO. + + sshj - The new SSH client with support for the latest crypto features. This is the default + when using the NED on NSO 5.6 or later. + + + - connection ssh host-key known-hosts-file + + Path to openssh formatted 'known_hosts' file containing valid host keys. + + + - connection ssh host-key public-key-file + + Path to openssh formatted public (.pub) host key file. + + + - connection ssh auth-key private-key-file + + Path to openssh formatted private key file. + + +# 3. ned-settings arris-cmts deprecated +--------------------------------------- + + Deprecated ned-settings. + + + - deprecated connection legacy-mode (default disabled) + + enabled - enabled. + + disabled - disabled. + + +# 4. ned-settings arris-cmts transaction +---------------------------------------- + + Transaction specific settings. + + + - transaction trans-id-method (default modeled-config) + + Select the method for calculating transaction-id. + + modeled-config - Use a snapshot of the data of only the modeled parts of running config for + calculation. + + full-config - Use a snapshot of the full running config for calculation. + + device-custom - Use a device custom method to get a value to use for trans-id. + + +# 5. ned-settings arris-cmts development_settings +------------------------------------------------- + + + - development_settings enable_device_save (default true) + + Set this to FALSE when you don't want the changes to be persistent on the device. USE WITH + CARE. + + +# 6. ned-settings arris-cmts proxy +---------------------------------- + + Configure NED to access device via a proxy. + + + - proxy remote-connection + + Connection type between proxy and device. + + ssh - ssh. + + telnet - telnet. + + serial - serial. + + + - proxy remote-address + + Address of host behind the proxy. + + + - proxy remote-port + + Port of host behind the proxy. + + + - proxy remote-name + + User name on the device behind the proxy. + + + - proxy remote-password + + Password on the device behind the proxy. + + + - proxy proxy-prompt + + Prompt pattern on the proxy host. + + + - proxy remote-ssh-args + + Additional arguments used to establish proxy connection. + + +# 7. ned-settings arris-cmts logger +----------------------------------- + + Settings for controlling logs generated. + + + - logger level (default info) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - logger java (default true) + + Toggle logs to be added to ncs-java-vm.log. + + +# 8. ned-settings arris-cmts developer +-------------------------------------- + + + - developer progress-verbosity (default debug) + + Maximum NED verbosity level which will get written in devel.log file. + + disabled - disabled. + + normal - normal. + + verbose - verbose. + + very-verbose - very-verbose. + + debug - debug. + + + - developer platform model + + Override device model name/number. + + + - developer platform name + + Override device name. + + + - developer platform version + + Override device version. + + + - developer trace-enable (default false) + + Enable developer tracing. WARNING: may choke NSO with large commits|systems. + + + - developer trace-timestamp (default false) + + Add timestamp from NED instance in trace messages for debug purpose. + + + - developer sync-from-verbose (default brief) + + Set info level for sync-from verbose output. + + brief - brief. + + full - full. + + +# 9. ned-settings arris-cmts proxy-settings +------------------------------------------- + + + - proxy-settings enable-connection (default false) + + enable proxy connection. + + + - proxy-settings address + + + - proxy-settings port + + + - proxy-settings user-name + + + - proxy-settings password + + + - proxy-settings expect-string + + +# 10. ned-settings arris-cmts ignore-error-messages +--------------------------------------------------- + + Add a list of valid error messages, or part of them. + + - arris-cmts ignore-error-messages + + - id + + diff --git a/arris-cmts/README.md b/arris-cmts/README.md new file mode 100644 index 00000000..c6ef2506 --- /dev/null +++ b/arris-cmts/README.md @@ -0,0 +1,719 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + ``` + + +# 1. General +------------ + + This document describes the arris-cmts NED. + + This document describes the NED for Arris CMTS E6000 devices. + + The NED connects to the device CLI using either SSH or Telnet. + Configuration is done by sending native CLI commands to the + device through the communication channel. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | See README.md for further info on how to configure the NED to | + | | | use netsim as a RESTCONF target. | + | | | | + | check-sync | no | - | + | | | | + | partial-sync-from | no | - | + | | | | + | live-status actions | no | - | + | | | | + | live-status show | no | - | + | | | | + | load-native-config | no | | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | E6000 | any | custom | - | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--arris-cmts-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-arris-cmts-1.0.1.signed.bin + > ./ncs-6.0-arris-cmts-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-arris-cmts-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-arris-cmts-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `arris-cmts-.`: + + ``` + > tar xfz ncs-6.0-arris-cmts-1.0.1.tar.gz + > ls -d */ + arris-cmts-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package arris-cmts-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package arris-cmts-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-arris-cmts-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package arris-cmts-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/arris-cmts-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-arris-cmts-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-arris-cmts-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install arris-cmts-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-arris-cmts-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-arris-cmts-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id arris-cmts-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-arris-cmts-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings arris-cmts logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings arris-cmts logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.arris_cmts \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings arris-cmts logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings arris-cmts logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + NONE + + +# 5. Built in live-status actions +--------------------------------- + + NONE + + +# 6. Built in live-status show +------------------------------ + + NONE + + +# 7. Limitations +---------------- + + 1. Cable dsg tunnel current way of working + + When entering and cable dsg tunnel configuration, for now, customer should enter both tunnel-group .. etc details, and tunnel classifier, + in the same transaction towards or from device. There are some defaults and some default device comportment that is not fully modeled yet. + + configure cable dsg tunnel 105 tunnel-group 11 client-id-list 3 mac-address 0100.5e1e.0102 service-class-name "" no + configure cable dsg tunnel 105 classifier 105 priority 1 source-network 10.32.210.241/32 dest-ip 239.30.1.2 dest-port-range 0-65535 include-in-dcd no + + 2. Using interface link-aggregate * isis wide-metric * level-1/2 + + When creating "interface link-aggregate * isis wide-metric * level-1/2" customer should enter both level-1 and level-2 components of wide-metric configuration point, + as device creates both indifferent of each one being created. + + The "no" form should be enterd with fully command, including the metric value as so : + - "no interface link-aggregate * isis wide-metric 1 level-1" (in case the metric value is set to 1), + the NED will automatically generate and default set command to be send to device of the following form: + - "interface link-aggregate * isis wide-metric 10 level-1" - deleting by setting to default. + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings arris-cmts logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings arris-cmts logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` diff --git a/brocade-ironware/README-ned-settings.md b/brocade-ironware/README-ned-settings.md new file mode 100644 index 00000000..5383f3e7 --- /dev/null +++ b/brocade-ironware/README-ned-settings.md @@ -0,0 +1,217 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/brocade-ironware/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/brocade-ironware/ + device + /ncs:/device/devices/device:/ned-settings/brocade-ironware/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings brocade-ironware + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings brocade-ironware + 2. connection + 3. dummy-connection + 4. deprecated + 5. logger + 6. live-status + ``` + + +# 1. ned-settings brocade-ironware +---------------------------------- + + Configure settings specific to the connection between NED and device. + + + - extended-parser (default auto) + + Make the brocade-ironware NED handle CLI parsing (i.e. transform the running-config from the + device to the model based config tree). + + auto - Uses turbo-mode when available, will use fastest availablemethod to load + data to NSO. If NSO doesn't support data-loading from CLI NED, robust-mode + is used. + + turbo-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO using maapi + setvalues call. + + turbo-xml-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO in XML + format. + + robust-mode - Makes the NED filter the configuration so that unmodeled content is removed + before being passed to the NSO CLI-engine. This protects against + configuration ending up at the wrong level when NSO CLI parser fallbacks + (which potentially can cause following config to be skipped). + + + - ironware-transaction-id-method + + Method of the brocade-ironware NED to use for calculating a transaction id. Typically used for + check-sync operations. + + config-hash - Use a snapshot of the running config for calculation. + + dir - Use the timestamp of the latest startup-configuration save time done via 'write + memory'for calculation. + + + - banner-delimeter-fastiron (default {) + + This is a single character delimiter used for setting a banner.This character must not be used + inside the banner text!. + + +# 2. ned-settings brocade-ironware connection +--------------------------------------------- + + Configure settings specific to the connection between NED and device. + + + - connection number-of-retries (default 0) + + Configure max number of retries the NED will try to connect to the device before giving up. + Default 0. + + + - connection time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + + - connection connector + + Change the default connector, e.g. 'ned-connector-default.json'. + + + - connection terminal width (default 200) + + Terminal width reported by SSH/TELNET upon connect. + + + - connection terminal height (default 24) + + Terminal height reported by SSH/TELNET upon connect. + + + - connection terminal println-mode (default onocr) + + This setting controls how the NED feeds lines, i.e. whether to + send carriage return and/or newline. Typically you may only need + to modify this setting if you are connecting to the device via a + terminal server. Four different methods are supported: + + default - System property line.separator default. + + ocrnl - Translate carriage return to newline. + + onocr - Translate newline to carriage return-newline. + + onlret - Newline performs a carriage return. + + + - connection ssh client + + Configure the SSH client to use. Relevant only when using the NED with NSO 5.6 or later. + + ganymed - The legacy SSH client. Used on all older versions of NSO. + + sshj - The new SSH client with support for the latest crypto features. This is the default + when using the NED on NSO 5.6 or later. + + +# 3. ned-settings brocade-ironware dummy-connection +--------------------------------------------------- + + Simulates a dummy connection, ONLY to be used with the 'load-native-config' feature, when the real + connection is not possible and the NED can't identify the Yang model. + + + - dummy-connection version (default ) + + Set version as below to choose the device type and the related Yang model. When set on empty, + a real connection is performed. + + +# 4. ned-settings brocade-ironware deprecated +--------------------------------------------- + + Deprecated ned-settings. + + + - deprecated live-status legacy-mode (default disabled) + + enabled - enabled. + + disabled - disabled. + + + - deprecated connection legacy-mode (default disabled) + + enabled - enabled. + + disabled - disabled. + + +# 5. ned-settings brocade-ironware logger +----------------------------------------- + + Settings for controlling logs generated. + + + - logger level (default debug) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - logger java (default true) + + Toggle logs to be added to ncs-java-vm.log. + + +# 6. ned-settings brocade-ironware live-status +---------------------------------------------- + + Configure NED settings related to live-status. + + + - live-status time-to-live (default 50) + + Define time-to-live for data fetched from the device via live-status.(default 50). + + diff --git a/brocade-ironware/README.md b/brocade-ironware/README.md new file mode 100644 index 00000000..c9badccd --- /dev/null +++ b/brocade-ironware/README.md @@ -0,0 +1,1424 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + 11. NETSIM testing + 12. How to properly config 'ip access-list standard/extended' on a Ruckus device + 13. The load-native-config feature + ``` + + +# 1. General +------------ + + This document describes the brocade-ironware NED. + + The brocade-ironware NED addresses the following devices: + - ServerIron ADX + - IronWare MLX + - FastIron Ruckus + - Brocade FastIron SX800/Foundry 2402/4802 + + The NED connects to the devices CLI using SSH or TELNET. + The configuration is done by sending native CLI commands to the device + through the communication channel. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | - | + | | | | + | check-sync | yes | - | + | | | | + | partial-sync-from | yes | - | + | | | | + | live-status actions | yes | - | + | | | | + | live-status show | yes | - | + | | | | + | load-native-config | yes | - | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Brocade ServerIron ADX | 12.5 | Server | - | + | | | Iron | | + | | | ADX | | + | | | | | + | Brocade MLX | 4.2 | IronWa | - | + | | | re | | + | | | | | + | Brocade MLX | 5.x | IronWa | - | + | | | re | | + | | | | | + | FastIron Ruckus | 10.1 | FastIr | - | + | | | on | | + | | | | | + | Brocade FastIron SX800 | 07.2.02hT3e3 | FastIr | - | + | | | on | | + | | | | | + | Foundry 2402-4802 | 04.0.00aTc1 | FastIr | - | + | | | on | | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--brocade-ironware-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-brocade-ironware-1.0.1.signed.bin + > ./ncs-6.0-brocade-ironware-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-brocade-ironware-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-brocade-ironware-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `brocade-ironware-.`: + + ``` + > tar xfz ncs-6.0-brocade-ironware-1.0.1.tar.gz + > ls -d */ + brocade-ironware-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package brocade-ironware-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package brocade-ironware-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-brocade-ironware-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package brocade-ironware-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/brocade-ironware-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-brocade-ironware-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-brocade-ironware-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install brocade-ironware-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-brocade-ironware-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-brocade-ironware-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id brocade-ironware-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-brocade-ironware-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings brocade-ironware logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings brocade-ironware logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.ironware \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings brocade-ironware logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings brocade-ironware logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + **SSHJ DEBUG LOGGING** + For issues related to the ssh connection it is often useful to enable full logging in the SSHJ ssh client. + This will make SSHJ print additional log entries in `$NSO_RUNDIR/logs/ncs-java-vm.log`: + +``` +admin@ncs(config)# java-vm java-logging logger net.schmizz.sshj level level-all +admin@ncs(config)# commit +``` + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + **Configuration for the ServerIron ADX device:** + + ``` + adx:snmp-server community test1 ro + adx:clock timezone us Arizona + adx:hostname host1 + + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name dev-1 + data hostname host1 + snmp-server community test1 ro + clock timezone us Arizona + } + } + ``` + Note: + - for the 'snmp-server community' entry, the related value(*test1*) will be encrypted by the device. + So, in order to have a synchronization between the NED and the device, the NED will decrypt the value, based on a mapping table stored on the NED side. + + + **Configuration for the IronWare MLX device:** + + ``` + mlx:vlan 1 name DEFAULT-VLAN + mlx:aaa authentication login default local + router mpls + vll s3 1 raw-mode cos 1 + vll-peer 18.1.1.1 + vlan 101 + tagged ethernet 1/1 + exit + exit + exit + policy-map CUST-100Mb + exit + mlx:interface ethernet 1/1 + rate-limit input vlan-id 101 policy-map CUST-100Mb + exit + + + admin@ncs(config-config)# show config + router mpls + vll s3 1 raw-mode cos 1 + vlan 101 + tagged ethernet 1/1 + exit + vll-peer 18.1.1.1 + exit + exit + mlx:interface ethernet 1/1 + exit + policy-map CUST-100Mb + exit + mlx:interface ethernet 1/1 + rate-limit input vlan-id 101 policy-map CUST-100Mb + exit + + + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name dev-1 + data router mpls + vll s3 1 raw-mode cos 1 + vlan 101 + tagged ethernet 1/1 + exit + vll-peer 18.1.1.1 + exit + exit + interface ethernet 1/1 + exit + policy-map CUST-100Mb + exit + interface ethernet 1/1 + rate-limit input vlan-id 101 policy-map CUST-100Mb + exit + } + } + ``` + + + **Configuration for the FastIron RUCKUS device:** + + Example 1: create a banner + + ``` + ruckus:vlan 123 name testVlan by port + tagged ethe 1/1/1 + untagged ethe 1/2/1 + exit + + ruckus:clock timezone europe GMT + ruckus:snmp-server community 2 $U2kyXj1k ro + ruckus:snmp-server community test2 ro + + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name dev-1 + data vlan 123 name testVlan by port + tagged ethe 1/1/1 + untagged ethe 1/2/1 + exit + clock timezone europe GMT + snmp-server community 2 $U2kyXj1k ro + snmp-server community test2 ro + } + } + ``` + + Note: + - for the 'snmp-server community', the values are also encrypted by the device, but only for the second format('test2'). + In this case, there is no mapping table available (compared to the ADX device), and the related encrypted values are stored into Operational DB(ODB). This is transparent for the user. + + Example setting a banner: + + ``` + admin@ncs(config-config)# banner motd "another banner\r\nfor test\r\nthird line" + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name dev-1 + data no banner motd + banner motd + another banner + for test + third line + } + } + + admin@ncs(config-config)# commit + Commit complete. + + admin@ncs(config-config)# compare-config + admin@ncs(config-config)# + ``` + + Important notes: + - the old banner is first deleted, otherwise the new text is appended to the old text of the banner + - new lines are given by the user with "\r\n" + - if a space is present at the beginning of a new line, the device will trim that space. Hence, please + don't start a new row with spaces, to avoid compare-config diffs. + - the device will use a delimiter character, for separating the banner's text. Its default value is "{". + The user must not use this value when setting the text. This delimiter is also configurable under ned-settings: + `admin@ncs(config-device-dev-1)# ned-settings brocade-ironware banner-delimeter-fastiron`. + Please don't forget to use `disconnect()` and `connect()` to enable the ned-setting. + For instance, if the delimiter is changed to "z", at the first occurence of letter "z" the text of the banner is considered finished + - the last line of the banner must not end with "\r\n". + + + Example 2: create snmp-server hosts + + ``` + admin@ncs(config-config)# ruckus:snmp-server host 1.2.3.4 version v1 community1 port 34 + admin@ncs(config-config)# ruckus:snmp-server host 1.2.3.4 version v2c community2 port 35 + admin@ncs(config-config)# ruckus:snmp-server host 1.2.3.5 version v3 auth aa port 19 + + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name dev-1 + data snmp-server host 1.2.3.4 version v1 community1 port 34 + snmp-server host 1.2.3.4 version v2c community2 port 35 + snmp-server host 1.2.3.5 version v3 auth aa port 19 + } + } + admin@ncs(config-config)# + ``` + On the device, entries above appear like: + + ``` + snmp-server host 1.2.3.4 version v1 ..... + snmp-server host 1.2.3.4 version v2c ..... + snmp-server host 1.2.3.5 version v3 auth aa port 19 + ``` + + Note: + - At sync-from, NED will check what are the CDB related data for "community" and "port" and will build the entire snmp-server host entries, similar as when they were created. This way, the compare-config diffs are avoided. + For those snmp-server host entries that are already present on the device, there is no deletion operation supported, since the device requires both community and port parameters to be provided at deletion. + For the version "v3", entries appear completely on the device. + The NED knows if an entry on the device is incomplete, by looking at the five dots `.....`. + + +# 5. Built in live-status actions +--------------------------------- + + The NED includes support for operational `show` commands and the following action can be used: + `devices device live-status exec show `. + + Example for the MLX device: + + ``` + admin@ncs(config)# devices device dev-1 live-status exec show version + result + System Mode: MLX + Chassis: NetIron 4-slot (Serial #: SA13075075, Part #: 35550-000B) + NI-X-SF Switch Fabric Module 1 (Serial #: SA1507ABCD, Part #: 31148-111F) + FE 1: Type fe200, Version 1 + Switch Fabric Module 1 Up Time is 100 days 18 hours 43 minutes 49 seconds + NI-X-SF Switch Fabric Module 2 (Serial #: SA15000000, Part #: 331148-111F) + FE 1: Type fe200, Version 22 + Switch Fabric Module 2 Up Time is 100 days 18 hours 43 minutes 49 seconds + ========================================================================== + SL M1: NI-MLX-MR Management Module Active (Serial #: SA46000000, Part #: 331148-111F): + Boot : Version 5.6.0T165 Copyright (c) 1996-2013 Brocade Communications Systems, Inc. + + ``` + + +# 6. Built in live-status show +------------------------------ + + Examples of running live-status commands for the MLX device: + + ``` + admin@ncs# show devices device dev-1 live-status interfaces + TYPE NAME IP ADDRESS MAC ADDRESS + ------------------------------------------------------------ + 10GigabitEthernet 1/1 - 0012.f290.1100 + 10GigabitEthernet 1/2 - 0012.f290.1101 + 10GigabitEthernet 1/3 - 0012.f290.1102 + 10GigabitEthernet 1/4 - 0012.f290.1103 + management 1 100.20.131.134/25 0012.f290.1100 + ``` + + The time for which values are stored in the memory is called ttl(time to live) and this is configurable under ned-settings: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings brocade-ironware live-status time-to-live 50 + ``` + + 50 represents the value of 50 seconds. + + +# 7. Limitations +---------------- + + ## 7.1 ipv6 access-list + + When configuring a permit or deny statement, a 'sequence' number will be assigned automatically by the device. + The default value is 10, and then 20, 30 and so on. In order to avoid compare-config diffs between the NED and the + target device, the user must provide explicitly the 'sequence' number when a permit or deny statement is set. + + Example: + Make sure to provide 'sequence' number like below: + + ``` + ipv6 access-list acl3 + sequence 3 permit ipv6 any host 2610:20:6f96:96::3 + sequence 8 permit ipv6 any host 2610:20:6f96:96::4 + exit + ``` + + instead of: + + ``` + ipv6 access-list acl3 + permit ipv6 any host 2610:20:6f96:96::3 + permit ipv6 any host 2610:20:6f96:96::4 + exit + ``` + ## 7.2 web-management https + + On the Ruckus device, when 'https' is set, the value is not visible (this is a default value). If the user will set again + the same value, the device returns a warning: "HTTPS already enabled". + In the ncs_cli, similar to the device, the 'https' is not visible. To be sure the 'https' value is set, the user can run the following command: + `show full | include https | details`. + If the output is: "ruckus:web-management https", it means the 'https' is already set. Trying to set it again will explicitly write the value + into CDB and will send the command to the device. + The user can disable the 'https' by running the following command: + `no ruckus:web-management https`. + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings brocade-ironware logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings brocade-ironware logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` + +# 11. NETSIM testing +------------------- + +NETSIM is configured to emulate the ADX device by default. To enable the MLX behaviour set the following env. variable +before building netsim: + `export NETSIM_BROCADE_DEV_TYPE=MLX` + +To enable the RUCKUS behavior, set the env. variable like this: + `export NETSIM_BROCADE_DEV_TYPE=RUCKUS` + +To enable the FastIron SX behavior, set the env. variable like this: + `export NETSIM_BROCADE_DEV_TYPE=FSX` + +# 12. How to properly config 'ip access-list standard/extended' on a Ruckus device +---------------------------------------------------------------------------------- + +Important notes: +- both 'ip access-list standard' and 'ip access-list extended' list are configurable in a similar manner. + Below you can find more details and examples referring to the 'extended' case +- rule 'sequence number' must always be provided by the user to avoid compare-config diffs +- on ruckus device, rules sequence numbers are ordered in ascending order +- in ncs_cli, the last created rule is added at the end of the list, no matter what the sequence number has, + if not chosen otherwise by the user +- to add a new rule with a smaller sequence number, the user must use the `insert` command and place the rule + in the right position with `after` or `before` existing sequence numbers +- if a new rule is not created after/before the correct existing rule sequnce number, the user can use the `move` command + or `sync-from` command to avoid out-of-sync issues +- to change an existing rule, the rule must be deleted first and then created withe new values. + Otherwise the device returns an error: "Error:ACL filter add failed! (Duplicate filter found. ErrorCode:2|2|1021)". + + +**Below, you can see an example on how to create an ip access-list extended:** + +``` +admin@ncs(config-config)# ruckus:ip access-list extended acle50 +admin@ncs(config-std-ipacl-acle50)# remark Denies all OSPF traffic, with logging +admin@ncs(config-std-ipacl-acle50)# sequence 7 deny ospf any any log +admin@ncs(config-std-ipacl-acle50)# remark new text for sequence 8 +admin@ncs(config-std-ipacl-acle50)# sequence 8 deny ip host 10.157.22.103 host 10.157.21.1 log +admin@ncs(config-std-ipacl-acle50)# exit + +admin@ncs(config-config)# show config +ruckus:ip access-list extended acle50 + remark Denies all OSPF traffic, with logging + sequence 7 deny ospf any any log + remark new text for sequence 8 + sequence 8 deny ip host 10.157.22.103 host 10.157.21.1 log +! + +admin@ncs(config-config)# commit dry-run outformat native +native { + device { + name dev-1 + data ip access-list extended acle50 + remark Denies all OSPF traffic, with logging + sequence 7 deny ospf any any log + remark new text for sequence 8 + sequence 8 deny ip host 10.157.22.103 host 10.157.21.1 log + ! + } +} + +``` + +One important thing to mention here is that the 'sequence number' must be provisioned by the user all the time. +Even if the device accepts rules without mentioning the sequence number, a sequence number will be added automatically +by the device itself. Hence, to avoid further compare-config diffs, the sequence number must be provided by the user from the beginning. + +If the user continues to add more rules to the 'acle50' access-list, then he must pay attention to the sequece number. +The device will display the rules in ascending order according to the sequence number. +On the other hand, in ncs_cli the last created rule is added at the end of the list, no matter what is the sequence number, if not chosen otherwise by the user. +For example, if the sequence number of the new rule is greater than all the sequence numbers of the existing rules, then the rule is simple created, as below: + +``` +admin@ncs(config-config)# ruckus:ip access-list extended acle50 +admin@ncs(config-std-ipacl-acle50)# remark this is another rule test +admin@ncs(config-std-ipacl-acle50)# sequence 22 deny ip host 10.157.21.105 host 10.157.22.10 log + +admin@ncs(config-std-ipacl-acle50)# show config +ruckus:ip access-list extended acle50 + remark this is another rule test + sequence 22 deny ip host 10.157.21.105 host 10.157.22.10 log +! + +admin@ncs(config-std-ipacl-acle50)# commit dry-run outformat native +native { + device { + name dev-1 + data ip access-list extended acle50 + remark this is another rule test + sequence 22 deny ip host 10.157.21.105 host 10.157.22.10 log + ! + } +} + +admin@ncs(config-std-ipacl-acle50)# commit +Commit complete. +``` + +As you can see below, the rule is added last: + +``` +admin@ncs(config-std-ipacl-acle50)# show full +devices device dev-1 + config + ruckus:ip access-list extended acle50 + remark Denies all OSPF traffic, with logging + sequence 7 deny ospf any any log + remark this is another rule test + sequence 22 deny ip host 10.157.21.105 host 10.157.22.10 log + ! + ! +! +``` + +**Create a new rule with sequence number smaller than the existing sequence numbers rules** + +Let's suppose the user wants to create a new rule with sequence number 10, which must come after existing 'sequence 7 deny ospf any any log', to respect the ascending order. + +To be able to do that, the user must use the `insert` command and move the rule with `before` or `after` keywords. +First, let's create the remark associated to the new rule and then the rule itself: + +``` +admin@ncs(config-std-ipacl-acle50)# top insert devices device dev-1 config ruckus:ip access-list extended acle50 remark Permits ICMP traffic from 10.157.22.x to 10.157.21.x before remark this is another rule test +``` + +This is how the access-list looks at the moment: + +``` +admin@ncs(config-std-ipacl-acle50)# show full +devices device dev-1 + config + ruckus:ip access-list extended acle50 + remark Denies all OSPF traffic, with logging + sequence 7 deny ospf any any log + remark Permits ICMP traffic from 10.157.22.x to 10.157.21.x + remark this is another rule test + sequence 22 deny ip host 10.157.21.105 host 10.157.22.10 log + ! + ! +! +``` + +Now, let's create the rule with sequence number 10: + +``` +admin@ncs(config-std-ipacl-acle50)# top insert devices device dev-1 config ruckus:ip access-list extended acle50 sequence 10 permit icmp 10.157.22.0 0.0.0.255 10.157.21.0 0.0.0.255 before remark this is another rule test + +admin@ncs(config-std-ipacl-acle50)# show config +ruckus:ip access-list extended acle50 + ! after sequence 7 deny ospf any any log + remark Permits ICMP traffic from 10.157.22.x to 10.157.21.x + sequence 10 permit icmp 10.157.22.0 0.0.0.255 10.157.21.0 0.0.0.255 +! + +admin@ncs(config-std-ipacl-acle50)# commit +Commit complete. +``` + +Using `insert` will help us move the new created rule on the right position. Now, our new list looks like below: + +``` +admin@ncs(config-std-ipacl-acle50)# show full +devices device dev-1 + config + ruckus:ip access-list extended acle50 + remark Denies all OSPF traffic, with logging + sequence 7 deny ospf any any log + remark Permits ICMP traffic from 10.157.22.x to 10.157.21.x + sequence 10 permit icmp 10.157.22.0 0.0.0.255 10.157.21.0 0.0.0.255 + remark this is another rule test + sequence 22 deny ip host 10.157.21.105 host 10.157.22.10 log + ! + ! +! +``` + +**What to do if a rule is not correctly created using `insert`** + +If the user creates a new rule with a sequence number smaller than the sequence numbers of the existing rules, the new rule is added last. +In our list above, let's create a new rule entry: + +``` +admin@ncs(config-std-ipacl-acle50)# sequence 20 permit ip any any + +admin@ncs(config-std-ipacl-acle50)# commit dry-run outformat native +native { + device { + name dev-1 + data ip access-list extended acle50 + sequence 20 permit ip any any + ! + } +} + +admin@ncs(config-std-ipacl-acle50)# commit +Commit complete. + +admin@ncs(config-std-ipacl-acle50)# show full +devices device dev-1 + config + ruckus:ip access-list extended acle50 + remark Denies all OSPF traffic, with logging + sequence 7 deny ospf any any log + remark Permits ICMP traffic from 10.157.22.x to 10.157.21.x + sequence 10 permit icmp 10.157.22.0 0.0.0.255 10.157.21.0 0.0.0.255 + remark this is another rule test + sequence 22 deny ip host 10.157.21.105 host 10.157.22.10 log + sequence 20 permit ip any any + ! + ! +! +``` + +As you can see, this is added last. Because the device orders rules in ascending order, we get compare-config diffs. + +To avoid this, the user has 2 options: + 1. use `sync-from` command to be in sync again and have the right order, i.e. sequence 20 after sequence 10 + 2. use the `move` command to bring the sequence 20 rule in the right position (after sequence 10) + +Example for use-case 2: + +``` +admin@ncs(config-std-ipacl-acle50)# top move devices device dev-1 config ruckus:ip access-list extended acle50 sequence 20 permit ip any any after sequence 10 permit icmp 10.157.22.0 0.0.0.255 10.157.21.0 0.0.0.255 + +admin@ncs(config-std-ipacl-acle50)# show config +ruckus:ip access-list extended acle50 + ! after sequence 10 permit icmp 10.157.22.0 0.0.0.255 10.157.21.0 0.0.0.255 + sequence 20 permit ip any any +! + +admin@ncs(config-std-ipacl-acle50)# commit dry-run outformat native +native { + device { + name dev-1 + data ip access-list extended acle50 + ! + } +} + + +admin@ncs(config-std-ipacl-acle50)# commit +Commit complete. +``` + +There is no new command to be sent to the device. Still, a connect to the device and a computation of the transaction ID are performed here. + +Now, looking at the ncs_cli, the list will have the rules in ascending order, similar to the device order: + +``` +admin@ncs(config-std-ipacl-acle50)# show full +devices device dev-1 + config + ruckus:ip access-list extended acle50 + remark Denies all OSPF traffic, with logging + sequence 7 deny ospf any any log + remark Permits ICMP traffic from 10.157.22.x to 10.157.21.x + sequence 10 permit icmp 10.157.22.0 0.0.0.255 10.157.21.0 0.0.0.255 + sequence 20 permit ip any any + remark this is another rule test + sequence 22 deny ip host 10.157.21.105 host 10.157.22.10 log + ! + ! +! +``` + +**How to update an existing rule sequence number** + +If a sequence number is needed to be changed, then the existing sequence number must be deleted first, otherwise the device returns +the following error: "Error:ACL filter add failed! (Duplicate filter found. ErrorCode:2|2|1021)". +Let's suppose we want to change the rule with sequence 10. + +Please check the example below: + +``` +admin@ncs(config-std-ipacl-acle50)# no remark Permits ICMP traffic from 10.157.22.x to 10.157.21.x +admin@ncs(config-std-ipacl-acle50)# no sequence 10 permit icmp 10.157.22.0 0.0.0.255 10.157.21.0 0.0.0.255 +``` + +This is our current list, before commit, where sequence 10 has been deleted: + +``` +admin@ncs(config-std-ipacl-acle50)# show full +devices device dev-1 + config + ruckus:ip access-list extended acle50 + remark Denies all OSPF traffic, with logging + sequence 7 deny ospf any any log + sequence 20 permit ip any any + remark this is another rule test + sequence 22 deny ip host 10.157.21.105 host 10.157.22.10 log + ! + ! +! + +admin@ncs(config-std-ipacl-acle50)# top insert devices device dev-1 config ruckus:ip access-list extended acle50 remark Permits ICMP traffic from 10.156.20.x to 10.155.22.x after sequence 7 deny ospf any any log + +admin@ncs(config-std-ipacl-acle50)# top insert devices device dev-1 config ruckus:ip access-list extended acle50 sequence 10 permit icmp 10.156.20.0 0.0.0.255 10.155.22.0 0.0.0.255 after remark Permits ICMP traffic from 10.156.20.x to 10.155.22.x +``` + +Current list after changed the rule with sequence 10: + +``` +admin@ncs(config-std-ipacl-acle50)# show full +devices device dev-1 + config + ruckus:ip access-list extended acle50 + remark Denies all OSPF traffic, with logging + sequence 7 deny ospf any any log + remark Permits ICMP traffic from 10.156.20.x to 10.155.22.x + sequence 10 permit icmp 10.156.20.0 0.0.0.255 10.155.22.0 0.0.0.255 + sequence 20 permit ip any any + remark this is another rule test + sequence 22 deny ip host 10.157.21.105 host 10.157.22.10 log + ! + ! +! +``` + +Below, you can see the changes and how they will sent to the device with `commit dry-run outformat native' command: + +``` +admin@ncs(config-std-ipacl-acle50)# show config +ruckus:ip access-list extended acle50 + no remark Permits ICMP traffic from 10.157.22.x to 10.157.21.x + no sequence 10 permit icmp 10.157.22.0 0.0.0.255 10.157.21.0 0.0.0.255 + ! after sequence 7 deny ospf any any log + remark Permits ICMP traffic from 10.156.20.x to 10.155.22.x + sequence 10 permit icmp 10.156.20.0 0.0.0.255 10.155.22.0 0.0.0.255 +! + +admin@ncs(config-std-ipacl-acle50)# commit dry-run outformat native +native { + device { + name dev-1 + data ip access-list extended acle50 + no remark Permits ICMP traffic from 10.157.22.x to 10.157.21.x + no sequence 10 + remark Permits ICMP traffic from 10.156.20.x to 10.155.22.x + sequence 10 permit icmp 10.156.20.0 0.0.0.255 10.155.22.0 0.0.0.255 + ! + } +} +admin@ncs(config-std-ipacl-acle50)# commit +Commit complete. +``` + +After changing the rule with sequence 10, we can see the changed rule ocuppies the right position: + +``` +admin@ncs(config-std-ipacl-acle50)# show full +devices device dev-1 + config + ruckus:ip access-list extended acle50 + remark Denies all OSPF traffic, with logging + sequence 7 deny ospf any any log + remark Permits ICMP traffic from 10.156.20.x to 10.155.22.x + sequence 10 permit icmp 10.156.20.0 0.0.0.255 10.155.22.0 0.0.0.255 + sequence 20 permit ip any any + remark this is another rule test + sequence 22 deny ip host 10.157.21.105 host 10.157.22.10 log + ! + ! +! +``` + + +# 13. The load-native-config feature +------------------------------------ + +The brocade-ironware NED supports the load-native-config feature. The user can check if a specific configuration is supported or not by the NED. + +Since the NED covers different YANG modules, the NED usually chooses the right YANG module when a `connect` operation is performed. +Since the load-native-config feature should be used without a real connection towards a device, a ned-setting has been added +to by-pass the `connect` and creates a dummy connection. + +In order to use that, the user must set the following ned-setting: +``` +admin@ncs(config)# devices device dev-1 ned-settings brocade-ironware dummy-connection version ? +Description: Set version as below to choose the device type and the related Yang model. When set on empty, a real connection is performed. +Possible completions: + + Brocade ServerIron ADX consider a dummy connection for the ServerIron ADX device + Brocade version 4.2 dummy connection for the MLX device version 4.2 or 5.x + Chassis FastIron dummy connection for the Foundry/FastIron SX800 device + Ruckus consider a dummy connection for a FastIron Ruckus device +admin@ncs(config)# devices device dev-1 ned-settings brocade-ironware dummy-connection version + +admin@ncs(config)# devices device dev-1 ned-settings brocade-ironware dummy-connection version Chassis\ FastIron +admin@ncs(config-device-dev-1)# commit +Commit complete. + +admin@ncs(config-device-dev-1)# disconnect +admin@ncs(config-device-dev-1)# connect +result false +info Failed to connect to device dev-1: transport closed +admin@ncs(config-device-dev-1)# *** ALARM connection-failure: Failed to connect to device dev-1: transport closed +admin@ncs(config-device-dev-1)# + +``` +Once the version of the device is set and the dummy connection is established, then the load-native-config command can be called. + +Please note that when the NED must connect to the real device, the ned-setting above must be set on empty string(""), which is the default value. + +The user can choose to load the configuration by providing a configuration string or to load the configuration from a file. +An important thing to take into account is that the configuration must be provided in the native format of the device. +The current devices contain an '\r\n' at the end of each line and each configuration contains, at the beginning, the string +'Current configuration:'. +Therefore, these must be provided by the user when the 'load-native-config' command is called. + +Examples: + - providing a string with device's native configuration + +``` +admin@ncs(config-device-dev-1)# load-native-config data "Current configuration:\r\n!\r\nvlan 1 name DEFAULT-VLAN\r\nno untagged ethe 1/1 to 1/2\r\n!\r\nhostname host\r\n" +admin@ncs(config-device-dev-1)# show config +devices device dev1 + config + mlx:vlan 1 name DEFAULT-VLAN + ! + mlx:hostname host + ! +! +``` + + - providing the configuration from a file + +``` +admin@ncs(config)# devices device dev-1 load-native-config file file-config.txt +``` + +In the 'file-config.txt' file, the configuration can be copied directly from the target device, i.e. get the output after running +the 'show running-config' command. diff --git a/casa-ccap/README-ned-settings.md b/casa-ccap/README-ned-settings.md new file mode 100644 index 00000000..f5634405 --- /dev/null +++ b/casa-ccap/README-ned-settings.md @@ -0,0 +1,473 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/casa-ccap/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/casa-ccap/ + device + /ncs:/device/devices/device:/ned-settings/casa-ccap/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings casa-ccap + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings casa-ccap + 2. meta-data + 3. proxy + 4. live-status + 4.1. auto-prompts + 5. developer + 6. connection + 7. get-device-config-settings + 8. apply-device-config-settings + 9. behaviour + 9.1. config-abort-warning + 10. logger + ``` + + +# 1. ned-settings casa-ccap +--------------------------- + + + - casa-ccap persist (default true) + + this option will disable the 'copy running config to start up config' command that's issued + after commit. + + + - casa-ccap trans-id-method (default config-hash) + + Configure how the NED shall calculate the transaction id. Typically used after each commit and + for check-sync operations. + + config-hash - Use a snapshot of the running config for calculation.(default). + + rollback-timestamp - Use the time stamp of the latest rollback checkpoint for + calculation. The system rollback feature must be on. + + last-modified-timestamp - Use the 'time last modified' time stamp generated by the ALU device + for calculation. Note, this time stamp is not available on all ALU + devices. See README. + + last-saved-timestamp - Use the 'time last saved' time stamp generated by the ALU device + for calculation. Note, this method is not reliable. See README. + + + - casa-ccap candidate-commit (default disabled) + + Make the NED use the candidate commit feature available on some ALU devices. + + disabled - Apply configuration the standard way, one-by-one. (default). + + enabled - Apply configuration into a candidate and then commit it.The candidate feature must + be enabled on the device. + + + - casa-ccap extended-parser (default auto) + + Make the cisco-aireos NED handle CLI parsing (i.e. transform the running-config from the + device to the model based config tree). + + disabled - Load configuration the standard way. + + turbo-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO using maapi + setvalues call. + + turbo-xml-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO in XML + format. + + robust-mode - Makes the NED filter the configuration so that unmodeled content is removed + before being passed to the NSO CLI-engine. This protects against + configuration ending up at the wrong level when NSO CLI parser fallbacks + (which potentially can cause following config to be skipped). + + auto - Uses turbo-mode when available, will use fastest availablemethod to load + data to NSO. If NSO doesn't support data-loading from CLI NED, robust-mode + is used. + + + - casa-ccap number-of-lines-to-send-in-chunk (default 100) + + Number of commands lines in a chunk sent by the casa-ccap NED to the device. Default is 100. A + higher number normally result in better performance but will also have negative impact on the + error handling. This command does also control the chunk sizes used in partial show bulk + mode. + + + - casa-ccap partial-show-method (default bulk-mode) + + Method to use when executing a partial show on the device (for instance when doing a 'commit + no-overwrite'). + + bulk-mode - The NED executes in a bulk mode fashion. Commands to display config on the + requested locations on the device are sent in chunks to minimize the round trip + time. The result is gathered when all commands have been sent to the device. + (default). + + walk-mode - The NED walks the config tree on the device step by step and extracts the + config on the requested locations. This method is much slower than bulk mode + but can be an alternative for devices that are not able to handle chunks of + commands properly. + + filter-mode - The NED fetches a full configuration dump from the device. It then filters out + everything except the requested the parts. The filtered config is then sent + back to NSO. +Note this method is always used when connected to a NETSIM device. + + +# 2. ned-settings casa-ccap meta-data +------------------------------------- + + Configure how the NED shall operate on the meta-data tags used in the YANG model. + + + - meta-data shutdown-before-set (default enabled) + + This operation makes the NED shutdown a parent before (given by path) setting the value of + this leaf. Enabled by default. + + enabled - enabled. + + disabled - disabled. + + + - meta-data redeploy (default enabled) + + This operation makes the NED redeploy another leaf (given by path) after setting the value of + this leaf. Enabled by default. + + enabled - enabled. + + disabled - disabled. + + +# 3. ned-settings casa-ccap proxy +--------------------------------- + + Configure NED to access device via a proxy. + + + - proxy remote-connection + + Connection type between proxy and device. + + ssh - ssh. + + telnet - telnet. + + serial - serial. + + + - proxy remote-address + + Address of host behind the proxy. + + + - proxy remote-port + + Port of host behind the proxy. + + + - proxy remote-name + + User name on the device behind the proxy. + + + - proxy remote-password + + Password on the device behind the proxy. + + + - proxy proxy-prompt + + Prompt pattern on the proxy host. + + + - proxy remote-ssh-args + + Additional arguments used to establish proxy connection. + + +# 4. ned-settings casa-ccap live-status +--------------------------------------- + + Configure NED settings related to live-status. + + + - live-status time-to-live (default 50) + + Define time-to-live for data fetched from the device via live-status.(default 50). + + +## 4.1. ned-settings casa-ccap live-status auto-prompts +------------------------------------------------------- + + Pre-stored answers to device prompting questions. + + - live-status auto-prompts + + - id + + List id, any string. + + - question + + Device question, regular expression. + + - answer + + Answer to device question. + + +# 5. ned-settings casa-ccap developer +------------------------------------- + + Contains settings used by the NED developers. + + + - developer load-from-file + + Make the NED load a file containing raw device config when doing sync-from. Does only work on + NETSIM targets. + + + - developer model + + Simulate a model number. + + + - developer version + + Simulate a version number. + + + - developer device-type (default netsim) + + Real or simulated device. + + netsim - netsim. + + device - device. + + + - developer progress-verbosity (default debug) + + Maximum NED verbosity level which will get written in devel.log file. + + disabled - disabled. + + normal - normal. + + verbose - verbose. + + very-verbose - very-verbose. + + debug - debug. + + + - developer platform model + + Override device model name/number. + + + - developer platform name + + Override device name. + + + - developer platform version + + Override device version. + + + - developer trace-enable (default false) + + Enable developer tracing. WARNING: may choke NSO with large commits|systems. + + + - developer trace-timestamp (default false) + + Add timestamp from NED instance in trace messages for debug purpose. + + + - developer sync-from-verbose (default brief) + + Set info level for sync-from verbose output. + + brief - brief. + + full - full. + + +# 6. ned-settings casa-ccap connection +-------------------------------------- + + Configure settings specific to the connection between NED and device. + + + - connection connector + + Change the default connector. Default 'ned-connector-default.json'. + + + - connection ssh client (default ganymed) + + Configure the SSH client to use. Relevant only when using the NED with NSO 5.6 or later. + + ganymed - The legacy SSH client. Used on all older versions of NSO. + + sshj - The new SSH client with support for the latest crypto features. This is the default + when using the NED on NSO 5.6 or later. + + + - connection ssh host-key known-hosts-file + + Path to openssh formatted 'known_hosts' file containing valid host keys. + + + - connection ssh host-key public-key-file + + Path to openssh formatted public (.pub) host key file. + + + - connection ssh auth-key private-key-file + + Path to openssh formatted private key file. + + + - connection number-of-retries (default 0) + + Configure max number of retries the NED will try to connect to the device before giving up. + Default 0. + + + - connection time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + +# 7. ned-settings casa-ccap get-device-config-settings +------------------------------------------------------ + + Configure how the NED shall read config from the device. + + + - get-device-config-settings method (default cli) + + The method to use. + + cli - Dump the running config using the 'admin display-config' CLI command + (default). + + sftp-transfer - Transfer the latest saved config from the device via the SFTP protocol. Note + that this method is not reliable. The latest saved config is not necessarily + the same as the running config. A NED configured for this will not be able to + detect out of band changes that have not been saved to file yet. + + + - get-device-config-settings file + + The name of the file containing latest saved config. + + + - get-device-config-settings save-running-config-first + + Make the NED automatically save the running config to the specified file before starting the + SFTP transfer. + + enabled - enabled. + + disabled - disabled. + + +# 8. ned-settings casa-ccap apply-device-config-settings +-------------------------------------------------------- + + Configure how the NED shall write config to the device. + + + - apply-device-config-settings method (default cli) + + The method to use. + + cli - Configure through the CLI (default). + + sftp-transfer - Transfer as file to the device via the SFTP protocol. Then apply the config + using the 'exec' command on the device. + + + - apply-device-config-settings file (default cf3:casa-ccap-tmp.cfg) + + The name of the temporary file to use when transferring the config (default: + 'cf3:casa-ccap-tmp.cfg'. + + +# 9. ned-settings casa-ccap behaviour +------------------------------------- + + NED specific behaviours. + + +## 9.1. ned-settings casa-ccap behaviour config-abort-warning +------------------------------------------------------------- + + Configure additional device warnings that shall be treated as errors and trigger an abort in the + NED. Enter as a regex. + + - behaviour config-abort-warning + + - warning + + Warning regular expression, e.g. .*interface.* does not exist.*. + + +# 10. ned-settings casa-ccap logger +----------------------------------- + + Settings for controlling logs generated. + + + - logger level (default info) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - logger java (default true) + + Toggle logs to be added to ncs-java-vm.log. + + diff --git a/casa-ccap/README.md b/casa-ccap/README.md new file mode 100644 index 00000000..194b7b57 --- /dev/null +++ b/casa-ccap/README.md @@ -0,0 +1,746 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + ``` + + +# 1. General +------------ + + This document describes the casa-ccap NED. + + This document describes the CLI NED for Casa Ccap C100G device. + The NED connects to the device CLI using SSH. + Configuration is done by sending native CLI commands to the + device through the communication channel. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | - | + | | | | + | check-sync | yes | - | + | | | | + | partial-sync-from | no | - | + | | | | + | live-status actions | yes | - | + | | | | + | live-status show | yes | - | + | | | | + | load-native-config | no | | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Casa C100G | 7.2.6.0 | Casa S | - | + | | | MM_8x1 | | + | | | 0G 7.2 | | + | | | .6.0 | | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--casa-ccap-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-casa-ccap-1.0.1.signed.bin + > ./ncs-6.0-casa-ccap-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-casa-ccap-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-casa-ccap-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `casa-ccap-.`: + + ``` + > tar xfz ncs-6.0-casa-ccap-1.0.1.tar.gz + > ls -d */ + casa-ccap-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package casa-ccap-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package casa-ccap-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-casa-ccap-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package casa-ccap-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/casa-ccap-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-casa-ccap-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-casa-ccap-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install casa-ccap-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-casa-ccap-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-casa-ccap-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id casa-ccap-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-casa-ccap-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings casa-ccap logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings casa-ccap logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.casaccap \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings casa-ccap logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings casa-ccap logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + NONE + + +# 5. Built in live-status actions +--------------------------------- + + 1. `config exec persist`: Saves the running config to start-up config by calling `copy running-config startup-config` + 2. `live-status exec any (args)`: It will send the entire set of parameters towards the device, exactly as they are + + +# 6. Built in live-status show +------------------------------ + + NONE + + +# 7. Limitations +---------------- + + 1. Rules for adding 'aggregate-address' list mode: + If the users want to add an 'aggregate-address' in the 'router bgp ****' mode, they will have to do it in the NED by entering the specific address-family mode + corresponding to where they want the 'aggregate-address' entry to be. The command will no work from the root of 'bgp router ****' as it is not clear in which + 'address-family' mode the device will add the new 'aggregate-address'. + + 2. Device rules: + ip access-list rules will have to be entered with their corresponding rule sequence number on permit/deny/remark line. + + 3. Live-status access-list resequence using format is as follows: + devices device + live-status EXEC any config ip access-list resequence + + 4. interface upstream * usage: + NED should be always used after sync from device. + Do not NO interface upstream. + Provide all device defaults on creation in same COMMIT/ROLLBACK initial transaction. + + 5. interface qam * usage: + NED should be always used after sync from device. + Do not NO interface qam. + Provide all device defaults on creation in same COMMIT/ROLLBACK initial transaction. + + 6. interface ofdma * usage: + NED should be always used after sync from device. + Do not NO interface ofdma. + Provide all device defaults on creation in same COMMIT/ROLLBACK initial transaction. + + 7. interface docsis-mac * router-advertisement life-time **ISSUE !! READ BEFORE USE !!**: + Default value of leaf is dependant on config parts that we have not implemented yet. Use with caution, once set, top rollback will create a `no * * * life-time` command which is not valid on device. If set, delete whole list entry, or remove on device by "setting to default" and re-sync-from. This can be fixed when we have all config, that "life-time" is dependant on, modeled. + + 8. load-balance execution rule * : + When modifying/creating upstream-threshold, both minimum and dynamic-minimum values should be set in the same command, as the device modifies/creates/removes dynamic-minimum when just minimum is entered at the end of the command + ``` + load-balance policy 1 + rule execution 1 + ``` + Should both be set or deleted whenever we want to create a rule-execution entry inside load-balance policy * . + Should both be deleted when we want to empty the load-balance policy node. Devices creates and removes the load-balance policy entry whenever we create/remove a rule execution entry. + + 9. snmp trap version * : + SNMP trap version instances are shown as their real value on the device using "verbose" command. + "no" version will be shown as non existant in NED. + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings casa-ccap logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings casa-ccap logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` diff --git a/ceragon-ip20/README-ned-settings.md b/ceragon-ip20/README-ned-settings.md new file mode 100644 index 00000000..25abe1df --- /dev/null +++ b/ceragon-ip20/README-ned-settings.md @@ -0,0 +1,616 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/ceragon-ip20/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/ceragon-ip20/ + device + /ncs:/device/devices/device:/ned-settings/ceragon-ip20/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings ceragon-ip20 + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings ceragon-ip20 + 2. developer + 3. proxy + 4. proxy-2 + 5. connection + 6. transaction + 7. console + 7.1. warning + 7.2. command + 7.3. pattern + 7.4. action + 7.4.1. state + 8. logger + ``` + + +# 1. ned-settings ceragon-ip20 +------------------------------ + + + - extended-parser (default auto) + + Make the NED handle CLI parsing (i.e. transform the running-config from the device to the + model based config tree). + + auto - Uses turbo-mode when available, will use fastest availablemethod to load + data to NSO. + + turbo-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO using maapi + setvalues call. + + turbo-xml-mode - The NED executes the whole command parsing by itself, completely bypassing + the NSO CLI parser. The configuration dump is transferred to NSO in XML + format. + + +# 2. ned-settings ceragon-ip20 developer +---------------------------------------- + + Contains settings used by the NED developers. + + + - developer load-from-file + + Make the NED load a file containing raw device config when doing sync-from. Does only work on + NETSIM targets. + + + - developer model-id + + Simulate a model number. + + + - developer os-major + + Simulate OS major number. + + + - developer os-minor (default 0) + + Simulate OS major number. + + + - developer device-type (default netsim) + + Real or simulated device. + + netsim - netsim. + + device - device. + + + - developer trace-enable (default false) + + Enable developer tracing. WARNING: may choke NSO with large commits|systems. + + + - developer trace-connection (default false) + + Enable connection tracing. WARNING: may choke NSO with IPC messages. + + + - developer trace-timestamp (default false) + + Add timestamp from NED instance in trace messages for debug purpose. + + + - developer progress-verbosity (default verbose) + + Maximum NED verbosity level which will get written in devel.log file. + + disabled - disabled. + + normal - normal. + + verbose - verbose. + + very-verbose - very-verbose. + + debug - debug. + + + - developer load-native-config allow-delete (default false) + + Enable this setting to be able to handle limited delete operations with 'load-native-config'. Please note that + not all syntax available on a real device works, some delete operations can not be parsed by the NED. Use the + 'verbose' flag to 'load-native-config' to see if delete commands can be parsed. Currently this is only supported + when 'extended-parser' is set to 'turbo-xml-mode' + + + - developer load-native-config delete-with-remove (default false) + + Enable this setting to use 'remove' instead of 'delete' when sending delete operations to NSO. This is useful when + doing delete commands for data that might not be present in CDB. Please note that deletes for missing data will still + be part of transaction, and will be sent to device. Use with care, and do proper testing to understand behaviour. + + + - developer platform model + + Override device model name/number. + + + - developer platform name + + Override device name. + + + - developer platform version + + Override device version. + + +# 3. ned-settings ceragon-ip20 proxy +------------------------------------ + + Configure NED to access device via a proxy. + + + - proxy remote-connection + + Connection type between ned, proxy and device. + + ssh - Start a new ssh client on proxy and connect to device (i.e. will launch + interactive shell on proxy). + + telnet - Start a new telnet client on proxy and connect to device (i.e. will launch + interactive shell on proxy). + + serial - Connect through a terminal server to device. + + ssh-direct - Direct forward to device using ned local ssh client (i.e. without shell on + proxy). + + telnet-direct - Direct forward to device using ned local telnet client (i.e. without shell on + proxy). + + + - proxy remote-address + + Address of host behind the proxy. + + + - proxy remote-port + + Port of host behind the proxy. + + + - proxy remote-name + + User name on the device behind the proxy. + + + - proxy remote-password + + Password on the device behind the proxy. + + + - proxy remote-secondary-password + + Second password (e.g. enable) on the device behind the proxy .WARNING MUST UPDATE connector + template to use!. + + + - proxy authgroup + + Authentication credentials for the device behind the proxy. + + + - proxy proxy-prompt + + Prompt pattern on the proxy host before connecting to device. + + + - proxy remote-ssh-args + + Additional arguments used to establish proxy connection. + + + - proxy auth-key private-key-file + + Path to openssh formatted private key file for doing public key auth to device behind proxy. + + + - proxy host-key-validation (default false) + + Set this to true to force host-key validation of device behind proxy. + + +# 4. ned-settings ceragon-ip20 proxy-2 +-------------------------------------- + + Configure NED to access device via a second proxy. + + + - proxy-2 remote-connection + + Connection type between ned, proxy and device. + + ssh - Start a new ssh client on proxy and connect to device (i.e. will launch + interactive shell on proxy). + + telnet - Start a new telnet client on proxy and connect to device (i.e. will launch + interactive shell on proxy). + + serial - Connect through a terminal server to device. + + ssh-direct - Direct forward to device using ned local ssh client (i.e. without shell on + proxy). + + telnet-direct - Direct forward to device using ned local telnet client (i.e. without shell on + proxy). + + + - proxy-2 remote-address + + Address of host behind the proxy. + + + - proxy-2 remote-port + + Port of host behind the proxy. + + + - proxy-2 remote-name + + User name on the device behind the proxy. + + + - proxy-2 remote-password + + Password on the device behind the proxy. + + + - proxy-2 remote-secondary-password + + Second password (e.g. enable) on the device behind the proxy .WARNING MUST UPDATE connector + template to use!. + + + - proxy-2 authgroup + + Authentication credentials for the device behind the proxy. + + + - proxy-2 proxy-prompt + + Prompt pattern on the proxy host before connecting to device. + + + - proxy-2 remote-ssh-args + + Additional arguments used to establish proxy connection. + + + - proxy-2 auth-key private-key-file + + Path to openssh formatted private key file for doing public key auth to device behind proxy. + + + - proxy-2 host-key-validation (default false) + + Set this to true to force host-key validation of device behind proxy. + + +# 5. ned-settings ceragon-ip20 connection +----------------------------------------- + + Configure settings specific to the connection between NED and device. + + + - connection ssh client + + Configure the SSH client to use. Relevant only when using the NED with NSO 5.6 or later. + + ganymed - The legacy SSH client. Used on all older versions of NSO. + + sshj - The new SSH client with support for the latest crypto features. This is the default + when using the NED on NSO 5.6 or later. + + + - connection ssh host-key known-hosts-file + + Path to openssh formatted 'known_hosts' file containing valid host keys. + + + - connection ssh host-key public-key-file + + Path to openssh formatted public (.pub) host key file. + + + - connection ssh auth-key private-key-file + + Path to openssh formatted private key file. + + + - connection number-of-retries (default 0) + + Configure max number of retries the NED will try to connect to the device before giving up. + Default 0. + + + - connection time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + + - connection character-set (default UTF-8) + + Character set to use for telnet session. + + + - connection commands meta-data + + Change the default connector. Default 'ned-connector.json'. + + + - connection commands initial-action + + Interactor action used to initialize a connection. + + + - connection commands awaken-console + + Command sent to awaken console during connection. + + + - connection commands send-delay (default 0) + + Delay in ms before sending a command during connection. + + + - connection commands expect-timeout (default 60000) + + Default limit in ms for waiting for command response. + + + - connection logger silent (default false) + + Toggle detailed logs to only written to store. + + + - connection terminal width (default 10000) + + + - connection terminal height (default 200) + + +# 6. ned-settings ceragon-ip20 transaction +------------------------------------------ + + Transaction specific settings. + + + - transaction trans-id-method (default modeled-config) + + Select the method for calculating transaction-id. + + modeled-config - Use a snapshot of the data of only the modeled parts of running config for + calculation. + + full-config - Use a snapshot of the full running config for calculation. + + device-custom - Use a device custom method to get a value to use for trans-id. + + + - transaction abort-on-diff (default false) + + Enable to detect diff immediately when config is applied (i.e. in commit/abort/revert). + If a diff is detected an exception is thrown, having the effect in commit that the transaction + is aborted (showing the diff). Note that this means some overhead in commit, where whole config + needs to be retrieved from device to do compare. It's a more exact way to detect OOB changes, + silently dropped config, and unknown side-effects to config (i.e. all causing a diff compared + to NSO state). In fact, it's the only method which guarantees that the config was actually + applied as desired. Since this incurs overhead it is strongly adviced that this feature is + only used during development. + + +# 7. ned-settings ceragon-ip20 console +-------------------------------------- + + Settings used while interacting with a device. + + + - console ignore-errors + + Flag indicating if errors should be ignored. + + + - console ignore-warnings + + Flag indicating if warnings should be ignored. + + + - console ignore-retries + + Flag indicating if retries should be ignored. + + + - console max-retries + + Maximum number of retries of a command. + + + - console retry-delay + + Number of ms before retrying a command. + + + - console send-delay + + Enable delay before sending commands. + + + - console expect-timeout + + Set default timeout for sending commands. + + + - console chunk-size + + Enable executing commands in chunks. + + + - console line-feed + + Overwrites default line-feed character. + + + - console obfuscate-secret + + Secrets will be obfuscated in trace & log files. + + +## 7.1. ned-settings ceragon-ip20 console extension warning +----------------------------------------------------------- + + Device warning regex entry list. Use to ignore warnings/errors etc. + + - console extension warning + + - warning + + Warning regular expression, e.g. vlan.* does not exist.*. + + +## 7.2. ned-settings ceragon-ip20 console extension command +----------------------------------------------------------- + + Extend available commands to send. + + - console extension command + + - name + + Key id of the command. + + - data + + Command. + + +## 7.3. ned-settings ceragon-ip20 console extension pattern +----------------------------------------------------------- + + Extend available patterns to expect. + + - console extension pattern + + - name + + Key id of the pattern. + + - data + + A regular expression. + + +## 7.4. ned-settings ceragon-ip20 console extension action +---------------------------------------------------------- + + Extend available actions to perform. + + - console extension action + + - name + + A name for the action. + + - init + + Command sent to intialize action. + + - flush + + Flush device buffer once action is completed. + + +### 7.4.1. ned-settings ceragon-ip20 console extension action state +------------------------------------------------------------------- + + Extend state machine with answers/questions to handle. + + - state + + - pattern + + Regular expression indicating action required. + + - method + + Method used to take action. + + reportInfo - reportInfo. + + reportError - reportError. + + reportWarning - reportWarning. + + sendCommand - sendCommand. + + sendSecret - sendSecret. + + sendRetry - sendRetry. + + recoverError - recoverError. + + - argument + + Additional info to method. + + - next (default DONE) + + State once action is taken. + + +# 8. ned-settings ceragon-ip20 logger +------------------------------------- + + Settings for controlling logs generated. + + + - logger level (default info) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - logger java (default true) + + Toggle logs to be added to ncs-java-vm.log. + + diff --git a/ceragon-ip20/README.md b/ceragon-ip20/README.md new file mode 100644 index 00000000..196f8512 --- /dev/null +++ b/ceragon-ip20/README.md @@ -0,0 +1,2124 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Configure the NED to use ssh multi factor authentication + ``` + + +# 1. General +------------ + + This document describes the ceragon-ip20 NED. + + + ## 1.1 General info and considerations. + + * **Ceragon IP-20 Products platform**: This NED addresses CERAGON Networks IP-20N chassis based devices with IP-20A, IP-20C configs in mind, but also IP-20D,E. + - We will reffer to them throughout this document simply as "the device" or "the Ceragon IP-20 device". + + * The Ceragon IP-20N device chassis run CeraOS 12.0 custom OS. Interaction with the device is done via CLI using SSH session encrypted with strong secure ciphers. Telnet seems to be also supported. + + * The device is very sensitive in general so use with caution the live-status commands. + + * Running a big chunk of the configurations into the CHASSIS can overload the port causing unpredictable errors or loss of device, so try to iterate through the commands and break down the commands logic in as many small chunks as possible. + + * A lot of commands can be interactive. Some commands reboot the device or restart the cards inserted into the chassis. + + ### - Please note that the NED does not support fully interactive interactions with the device CLI. + * Multiple commands are interactive. + * For those, a defaulted yes/no confirmation behavior will be implemented, so that when a yes/no prompt occurs, NED is supposed to respond with **yes** at all times. + + + ## 1.2 Top configurable modules + + * platform management + - activation-key + - management + - shelf-manager + - if-manager + * ethernet + - generalcfg + - qos + * radio + - interfaces + - rf + - lcl-rmt-ch + - xpic + + ## 1.3 Asynchronous or interactive commands + + * A lot of commands, as mentioned earlier, are either interactive or asynchronous commands. Some of them create significant delays, significant but impossible to follow, as we don't get any indicator on the remaining/elapsed or potential delay time. + + Example of interactive commands in device SSH CLI syntax that could be used via live-status exec any commands: + + + ``` + platform management set-to-default + Are you sure? (yes/no):yes + ``` + + + + ``` + radio [1/1]>mrmc set acm-support script-id 5710 modulation fixed profile 9 + This operation may reset the radio interface and affect traffic. + Are you sure? (yes/no):yes + error number 9: SW error: thread context + ``` + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | no | | + | | | | + | check-sync | no | check-sync using trans-id DISABLED by default | + | | | | + | partial-sync-from | no | This feature is supported by filtering the needed config from a | + | | | full show (device does not support partial show) | + | | | | + | live-status actions | yes | Supports live staus exec any command | + | | | | + | live-status show | no | The NED does not implement TTL-based data | + | | | | + | load-native-config | no | | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | IP20-N | 12.0 | CeraOS | IP20 N and IP20 A Chassis | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--ceragon-ip20-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-ceragon-ip20-1.0.1.signed.bin + > ./ncs-6.0-ceragon-ip20-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-ceragon-ip20-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-ceragon-ip20-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `ceragon-ip20-.`: + + ``` + > tar xfz ncs-6.0-ceragon-ip20-1.0.1.tar.gz + > ls -d */ + ceragon-ip20-cli-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package ceragon-ip20-cli-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package ceragon-ip20-cli-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-ceragon-ip20-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package ceragon-ip20-cli-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/ceragon-ip20-cli-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-ceragon-ip20-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-ceragon-ip20-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install ceragon-ip20-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-ceragon-ip20-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-ceragon-ip20-cli-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type cli ned-id ceragon-ip20-cli-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + ``` + admin@ncs(config)# devices device dev-1 protocol + ``` + + - If configured protocol is ssh, do fetch the host keys now: + + ``` + admin@ncs(config)# devices device dev-1 ssh fetch-host-keys + ``` + + # Initial configuration requirements + ## Please make sure the following MANDATORY ned-settings are properly configured with remote-name as needed for logging in. + + ``` + ned-settings ceragon-ip20 connection remote-name login_username + ``` + + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-ceragon-ip20-cli-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings ceragon-ip20 logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings ceragon-ip20 logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.ceragonip20 \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings ceragon-ip20 logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings ceragon-ip20 logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + **SSHJ DEBUG LOGGING** + For issues related to the ssh connection it is often useful to enable full logging in the SSHJ ssh client. + This will make SSHJ print additional log entries in `$NSO_RUNDIR/logs/ncs-java-vm.log`: + +``` +admin@ncs(config)# java-vm java-logging logger net.schmizz.sshj level level-all +admin@ncs(config)# commit +``` + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + + ## 4.1 Current Yang schema structure of the model adopted: + + - /config/platform *: + ``` + module: tailf-ned-ceragon-ip20 + +--rw platform + | +--rw activation-key + | | +--rw key? string + | +--rw management + | | +--rw protection + | | | +--rw admin? enumeration + | | +--rw system-name + | | | +--rw name? string + | | +--rw system-location + | | | +--rw name? string + | | +--rw system-contact + | | +--rw name? string + | +--rw shelf-manager + | | +--rw shelves* [slot] + | | +--rw slot uint16 + | | +--rw slot-name? string + | | +--rw admin? enumeration + | | +--rw expected-cardtype? string + | +--rw if-manager + | +--rw interfaces* [interface-type slot port] + | +--rw interface-type string + | +--rw slot uint16 + | +--rw port union + | +--rw admin? enumeration + ``` + + - /config/ethernet * : + + ``` + +--rw ethernet + | +--rw generalcfg + | | +--rw mru + | | +--rw size? uint32 + | +--rw qos + | | +--rw port-priority-profile-tbl* [profile-id] + | | +--rw profile-id uint16 + | | +--rw cos0 + | | | +--rw cos0-priority? string + | | | +--rw description? string + | | +--rw cos1 + | | | +--rw cos1-priority? string + | | | +--rw description? string + | | +--rw cos2 + | | | +--rw cos2-priority? string + | | | +--rw description? string + | | +--rw cos3 + | | | +--rw cos3-priority? string + | | | +--rw description? string + | | +--rw cos4 + | | | +--rw cos4-priority? string + | | | +--rw description? string + | | +--rw cos5 + | | | +--rw cos5-priority? string + | | | +--rw description? string + | | +--rw cos6 + | | | +--rw cos6-priority? string + | | | +--rw description? string + | | +--rw cos7 + | | +--rw cos7-priority? string + | | +--rw description? string + | +--rw interfaces* [type slot port] + | | +--rw type enumeration + | | +--rw slot uint16 + | | +--rw port string + | | +--rw description? string + | | +--rw media-type + | | | +--rw state? enumeration + | | +--rw autoneg + | | | +--rw state? enumeration + | | +--rw classification + | | | +--rw default-cos + | | | | +--rw state? uint16 + | | | +--rw type_802-1p + | | | | +--rw state? enumeration + | | | +--rw ip-dscp + | | | | +--rw state? enumeration + | | | +--rw mpls + | | | +--rw state? enumeration + | | +--rw priority + | | +--rw profile-id? string + | +--rw service* [sid type] + | +--rw sid uint16 + | +--rw type string + | +--rw admin? string + | +--rw evc-id? string + | +--rw description? string + | +--rw sp* [spid] + | +--rw spid string + | +--rw sp-type? string + | +--rw int-type? string + | +--rw interface? string + | +--rw slot? string + | +--rw port? string + | +--rw vlan? string + | +--rw sp-name? string + ``` + + - /config/radio * + ``` + +--rw radio + +--rw interfaces* [slot port] + | +--rw slot uint16 + | +--rw port uint16 + | +--rw rf + | | +--rw tx-level? uint32 + | | +--rw rx-frequency? uint32 + | | +--rw tx-frequency? uint32 + | | +--rw mute + | | | +--rw admin? enumeration + | | +--rw adjacent-channel? enumeration + | +--rw lcl-rmt-ch + | +--rw link-id? uint32 + +--rw xpic* [group] + +--rw group uint16 + +--rw carrier1 + | +--rw radio? uint16 + | +--rw port? uint16 + +--rw carrier2 + | +--rw radio? uint16 + | +--rw port? uint16 + +--rw admin-mode? enumeration + ``` + + + + ## 4.2 Sample config + + - Below config snippet shows the CDB config format in NSO Cisco Style CLI content dump. + - After configuring the device, one must run sync-from, to collect all data existing on the device configured. + + + ``` + config + platform activation-key key 1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ + platform management protection admin disable + platform management system-location name LocationName + platform management system-contact name ContactName + platform shelf-manager slot 1 + admin enable + expected-cardtype TCC-U + ! + platform shelf-manager slot 2 + admin disable + expected-cardtype Cleared + ! + platform shelf-manager slot 3 + slot-name "Link 2" + admin enable + expected-cardtype RIC-D + ! + platform if-manager interface-type ABConRFU slot 3 port 1 admin up + platform if-manager interface-type LinkBonding slot 1 port 1 admin up + platform if-manager interface-type ethernet slot 1 port 2 admin down + platform if-manager interface-type management slot 1 port 1 admin up + platform if-manager interface-type radio slot 3 port 1 admin up + platform if-manager interface-type synch slot 1 port 1 admin down + ethernet generalcfg mru size 1234 + ethernet qos port-priority-profile-tbl 9 cos0-priority 1 description "cos0 priority" cos1-priority 2 description "cos1 priority" cos2-priority 2 description "cos2 priority" cos3-priority 2 description "cos3 priority" cos4-priority 2 description "cos4 priority" cos5-priority 3 description "cos5 priority" cos6-priority 3 description "cos6 priority" cos7-priority 4 description singleWord + ethernet interfaces eth slot 1 port 1 + media-type state sfp + autoneg state off + classification default-cos state 2 + classification type_802-1p state trust + classification ip-dscp state trust + classification mpls state trust + priority profile-id 9 + ! + ethernet interfaces eth slot 1 port 2 + media-type state sfp + autoneg state off + classification default-cos state 2 + classification type_802-1p state trust + classification ip-dscp state trust + classification mpls state trust + priority profile-id 9 + ! + ethernet service sid 1 type p2p admin operational evc-id 123 description N.A. + ethernet service sid 2 type p2p admin operational evc-id 1234 description N.A. + ethernet service sid 2 type p2p sp spid 1 sp-type sap int-type eth interface dot1q slot 1 port 1 vlan 1234 sp-name N.A. + radio slot 1 port 1 + rf tx-level 1 + rf mute admin on + rf adjacent-channel disable + lcl-rmt-ch link-id 123 + ! + radio xpic group 1 radio 3 port 1 radio 3 port 2 admin-mode enable + ! + ! + ``` + + ## 4.3 Short Configuration summary samples from NSO CLI/NED towards the device: + + ### !!! Important considerations to take into account: + + - The Ceragon IP-20 device CLI config structure does **not** allow traditional operations to be performed on the entire data set, such as Create, Delete, Set|Modify, Show. + - Some modules support only SET|Modify, as they are always pre-populated when the hardware cards are inserted + - Some modules support Create/Delete as well. + - In the below examples we will cover the currently supported operation types. If we have one point on which we perform only SET, it means create/delete are not supported. + - Either way, if an illegal operation is tried, either the device or the NED will reject the operation and an exception will be thrown. + + + + ### 4.3.1 Configuring /data/platform/activation-key + + - Configure new activation key + ``` + devices device + config + platform activation-key key + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + platform { + activation-key { + - key ; + + key ; + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data platform activation-key set key string + } + } + ``` + + + ### 4.3.2 Configuring /data/platform/management + + - Configure/set params under platform/management : + - admin + - system-location name + - system-contact name + + ``` + devices device + config + platform management protection admin disable + platform management system-location name + platform management system-contact name "" + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + platform { + management { + protection { + - admin disable; + + admin enable; + } + system-location { + - name ; + + name ; + } + system-contact { + - name ; + + name ""; + } + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data platform management protection set admin enable + platform management system-location set name + platform management system-contact set name "" + } + } + admin@ncs(config-config)# commit + ``` + + + + ### 4.3.3 Configuring /data/platform/if-manager interface-type * slot * port * admin + + - Set admin down + ``` + devices device + config + platform if-manager interface-type ethernet slot 1 port 1 admin down + ! + ! + ``` + + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + platform { + if-manager { + interfaces ethernet 1 1 { + - admin up; + + admin down; + } + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data platform if-manager set interface-type ABConRFU slot 3 port 1 admin down + } + } + admin@ncs(config-config)# commit + ``` + + ### 4.3.4 Configuring /data/platform/shelf-manager{slot *} params + + - Configure|set /data/platform/shelf-manager{slot *}/admin enable|disable + - Configure|set /data/platform/shelf-manager{slot *}/expected-cardtype |Cleared + ``` + devices device + config + platform shelf-manager slot 1 + admin enable + expected-cardtype Cleared + ! + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + platform { + shelf-manager { + shelves 1 { + - admin enable; + + admin disable; + - expected-cardtype TCC-U; + + expected-cardtype Cleared; + } + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data platform shelf-manager admin set slot 1 state disable + platform shelf-manager expected-cardtype set slot 1 type Cleared + } + } + admin@ncs(config-config)# commit + ``` + + + ### 4.3.5 Configuring /ethernet/generalcfg/mru/size + + - Configure|set + ``` + devices device + config + ethernet generalcfg mru size 2000 + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + ethernet { + generalcfg { + mru { + - size 2000; + + size 1234; + } + } + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data ethernet generalcfg mru set size 1234 + } + } + admin@ncs(config-config)# commit + ``` + + + + ### 4.3.6 Configuring /ethernet/qos/port-priority-profile-tbl + + - Configure|set **Existing** /ethernet/qos/port-priority-profile-tbl * cos[0-7] priority or description: + - Configuring any update on the priority table, on the existing priority table, will generate a full priorty table update: + - Example below, update cos0-priority to 2 and description to "best effort updated" + + ``` + devices device + config + ethernet qos port-priority-profile-tbl 9 cos0-priority 2 description "best effort updated" cos1-priority 2 description "data service 4" cos2-priority 2 description "data service 3" cos3-priority 2 description "data service 2" cos4-priority 2 description "data service 1" cos5-priority 3 description "real time 2" cos6-priority 3 description "real time 1" cos7-priority 4 description management + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + ethernet { + qos { + port-priority-profile-tbl 9 { + cos0 { + - cos0-priority 1; + + cos0-priority 2; + - description "best effort"; + + description "best effort updated"; + } + } + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data ethernet qos port-priority-profile-tbl edit profile-id 9 cos0-priority 2 description "best effort updated" cos1-priority 2 description "data service 4" cos2-priority 2 description "data service 3" cos3-priority 2 description "data service 2" cos4-priority 2 description "data service 1" cos5-priority 3 description "real time 2" cos6-priority 3 description "real time 1" cos7-priority 4 description management + } + } + admin@ncs(config-config)# commit + ``` + + + ### 4.3.7 CREATE AND DELETE /ethernet/qos/port-priority-profile-tbl full profiles + --- + + + ### 4.3.7.1 CREATE /ethernet/qos/port-priority-profile-tbl profile 1 + - Create /ethernet/qos/port-priority-profile-tbl profile 1 + + ``` + devices device + config + ethernet qos port-priority-profile-tbl 1 cos0-priority 1 description "test description 1" cos1-priority 2 description "test description 2" cos2-priority 2 description "test description 3" cos3-priority 2 description "test description 4" cos4-priority 2 description "test description 5" cos5-priority 3 description "test description 6" cos6-priority 3 description "test description 7" cos7-priority 4 description single_word_no_whitespace_desc + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + ethernet { + qos { + + port-priority-profile-tbl 1 { + + cos0 { + + cos0-priority 1; + + description "test description 1"; + + } + + cos1 { + + cos1-priority 2; + + description "test description 2"; + + } + + cos2 { + + cos2-priority 2; + + description "test description 3"; + + } + + cos3 { + + cos3-priority 2; + + description "test description 4"; + + } + + cos4 { + + cos4-priority 2; + + description "test description 5"; + + } + + cos5 { + + cos5-priority 3; + + description "test description 6"; + + } + + cos6 { + + cos6-priority 3; + + description "test description 7"; + + } + + cos7 { + + cos7-priority 4; + + description single_word_no_whitespace_desc; + + } + + } + } + } + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data ethernet qos port-priority-profile-tbl add profile-id 1 cos0-priority 1 description "test description 1" cos1-priority 2 description "test description 2" cos2-priority 2 description "test description 3" cos3-priority 2 description "test description 4" cos4-priority 2 description "test description 5" cos5-priority 3 description "test description 6" cos6-priority 3 description "test description 7" cos7-priority 4 description single_word_no_whitespace_desc + } + } + admin@ncs(config-config)# commit + ``` + --- + + ### 4.3.7.2 DELETE /ethernet/qos/port-priority-profile-tbl profile 1 + + - Delete /ethernet/qos/port-priority-profile-tbl profile 1 + + ``` + devices device + config + no ethernet qos port-priority-profile-tbl 1 + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + ethernet { + qos { + - port-priority-profile-tbl 1 { + - cos0 { + - cos0-priority 1; + - description "test description 1"; + - } + - cos1 { + - cos1-priority 2; + - description "test description 2"; + - } + - cos2 { + - cos2-priority 2; + - description "test description 3"; + - } + - cos3 { + - cos3-priority 2; + - description "test description 4"; + - } + - cos4 { + - cos4-priority 2; + - description "test description 5"; + - } + - cos5 { + - cos5-priority 3; + - description "test description 6"; + - } + - cos6 { + - cos6-priority 3; + - description "test description 7"; + - } + - cos7 { + - cos7-priority 4; + - description single_word_no_whitespace_desc; + - } + - } + } + } + } + } + } + } + } + + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data ethernet qos port-priority-profile-tbl delete profile-id 1 + } + } + admin@ncs(config-config)# commit + ``` + + + + ### 4.3.8 Updating /ethernet/interfaces/eth{slot * port *} + + - Configure|set /ethernet/interfaces/eth{slot * port *}a/ * parameters: + - media-type state + - autoneg state on|off + - classification default-cos state <0-9> + - classification type_802-1p state trust|un-trust + - classification ip-dscp state trust|un-trust + - classification mpls state trust|un-trust + - priority profile-id <0-9> + + + - Configure ethernet interfaces slot 1 port 1 params: + ``` + devices device + config + ethernet interfaces eth slot 1 port 1 + media-type state auto-type + autoneg state on + classification default-cos state 1 + classification type_802-1p state un-trust + classification ip-dscp state un-trust + classification mpls state un-trust + priority profile-id 1 + ! + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + ethernet { + interfaces eth 1 1 { + media-type { + - state sfp; + + state auto-type; + } + autoneg { + - state off; + + state on; + } + classification { + default-cos { + - state 2; + + state 1; + } + type_802-1p { + - state trust; + + state un-trust; + } + ip-dscp { + - state trust; + + state un-trust; + } + mpls { + - state trust; + + state un-trust; + } + } + priority { + - profile-id 9; + + profile-id 1; + } + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + admin@ncs(config-interfaces-eth/1/1)# commit dry-run outformat native + native { + device { + name ip20-26 + data ethernet interfaces eth slot 1 port 1 + media-type state set auto-type + autoneg state set on + classification set default-cos state 1 + classification set type_802-1p state un-trust + classification set ip-dscp state un-trust + classification set mpls state un-trust + priority set profile-id 1 + exit + } + } + admin@ncs(config-config)# commit + ``` + + + - Please note that parameters under ethernet interfaces eth slot * port * can only be SET! + - Delete, Create operations are NOT supported by device. + + + ### 4.3.9 Configuring /ethernet/service {sid * type *} + + ### 4.3.9.1 Create new ethernet service. + + - Create new service, without SP: + ``` + devices device + config + ethernet service sid 1 type p2p admin operational evc-id 122 description N.A. + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + ethernet { + + service 1 p2p { + + admin operational; + + evc-id 122; + + description N.A.; + + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data ethernet service sid 1 type p2p admin operational evc-id 122 description N.A. + } + } + admin@ncs(config-config)# commit + ``` + + ### 4.3.9.2 Create new ethernet service with SP. + + - First line of config, with params admin, evc-id, description addresses the parameters of the ethernet service with sid 1. + - Second line of config, starting with **sp spid 1** is addressing the Service Point associated to ethernet service with sid 1: + ``` + devices device + config + ethernet service sid 1 type p2p admin operational evc-id 122 description N.A. + ethernet service sid 1 type p2p sp spid 1 sp-type sap int-type eth interface dot1q slot 1 port 1 vlan 1020 sp-name N.A. + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + ethernet { + + service 1 p2p { + + admin operational; + + evc-id 122; + + description N.A.; + + sp 1 { + + sp-type sap; + + int-type eth; + + interface dot1q; + + slot 1; + + port 1; + + vlan 1020; + + sp-name N.A.; + + } + + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data ethernet service add type p2p sid 1 admin operational evc-id 122 description N.A. + + ethernet service sid 1 + sp spid 1 sp-type sap int-type eth interface dot1q slot 1 port 1 vlan 1020 sp-name N.A. + exit + } + } + admin@ncs(config-config)# commit + ``` + + - In this case, we will first create the service and only after we will add the SP point by entering into ethernet service sid mode. + + + + + ### 4.3.9.3 Delete only ethernet service SP under an existing ethernet service. + + ``` + devices device + config + no ethernet service sid 1 type p2p sp spid 1 + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + ethernet { + service 1 p2p { + - sp 1 { + - sp-type sap; + - int-type eth; + - interface dot1q; + - slot 1; + - port 1; + - vlan 1020; + - sp-name N.A.; + - } + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data ethernet service sid 1 + sp delete spid 1 + exit + } + } + admin@ncs(config-config)# commit + ``` + + + + + + + + + + + ### 4.3.9.4 Delete only ethernet service which does not have any SPs associated: + + ``` + devices device + config + no ethernet service sid 1 type p2p + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + ethernet { + - service 1 p2p { + - admin operational; + - evc-id 122; + - description N.A.; + - } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data ethernet service delete type p2p sid 1 + } + } + admin@ncs(config-config)# commit + ``` + + + + + + + + + + ### 4.3.9.5 Delete an ethernet service HAS at least 1 SP associated: + + - Starting from below config: + ``` + admin@ncs(config-device-ip20-26)# show full config ethernet service sid 1 type p2p + devices device ip20-26 + config + ethernet service sid 1 type p2p admin operational evc-id 122 description N.A. + ethernet service sid 1 type p2p sp spid 1 sp-type sap int-type eth interface dot1q slot 1 port 1 vlan 1020 sp-name N.A. + ! + ! + ``` + + - We delete the top level ethernet/service : + + ``` + devices device + config + no ethernet service sid 1 type p2p + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + ethernet { + - service 1 p2p { + - admin operational; + - evc-id 122; + - description N.A.; + - sp 1 { + - sp-type sap; + - int-type eth; + - interface dot1q; + - slot 1; + - port 1; + - vlan 1020; + - sp-name N.A.; + - } + - } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data ethernet service sid 1 + sp delete spid 1 + exit + ethernet service delete type p2p sid 1 + } + } + admin@ncs(config-config)# commit + ``` + + - As you may notice, we have to enter service mode first, delete the SP, then exit and delete the service itself. + + + + + + + + + ### 4.3.10 Configuring /radio/{slot * port *} + + - Configure radio slot * port * params. + - Consider initial config: + ``` + devices device + config + radio slot 1 port 1 + rf tx-level 1 + rf mute admin on + rf adjacent-channel disable + lcl-rmt-ch link-id 123 + ! + ! + ! + ``` + + ``` + devices device + config + radio slot 1 port 1 + rf tx-level 2 + rf mute admin off + rf adjacent-channel enable + lcl-rmt-ch link-id 456 + ! + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + radio { + interfaces 1 1 { + rf { + - tx-level 1; + + tx-level 2; + mute { + - admin on; + + admin off; + } + - adjacent-channel disable; + + adjacent-channel enable; + } + lcl-rmt-ch { + - link-id 123; + + link-id 456; + } + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data radio slot 1 port 1 + rf set tx-level 2 + rf mute set admin off + rf adjacent-channel enable + lcl-rmt-ch link-id set 456 + exit + } + } + admin@ncs(config-config)# commit + ``` + + + + + + ### 4.3.11 Configuring /radio/xpic/group * + + + ### 4.3.11.1 Create /radio/xpic/group * without admin-mode + + - Create radio xpic group without mentioning any admin-mode + - It will create a admin-mode disabled radio xpic group. + - Ideally it should be enabled separately, after a delay. + ``` + devices device + config + radio xpic group 2 radio 4 port 1 radio 5 port 2 + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + radio { + + xpic 2 { + + carrier1 { + + radio 4; + + port 1; + + } + + carrier2 { + + radio 5; + + port 2; + + } + + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the command that will be issued + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data radio xpic create group 2 radio 4 port 1 radio 5 port 2 + } + } + admin@ncs(config-config)# commit + ``` + + + + ### 4.3.11.2 Create /radio/xpic/group * WITH explicit admin-mode enable|disable + + - Create radio xpic group specifying any admin-mode state will be broken down in two step commands + - It will create a radio xpic group with admin-mode which is disabled by default. + - Ideally they should be enabled/disabled in the service logic with a delay in between. + ``` + devices device + config + data radio xpic create group 2 radio 4 port 1 radio 5 port 2 admin-mode enable + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + radio { + + xpic 2 { + + carrier1 { + + radio 4; + + port 1; + + } + + carrier2 { + + radio 5; + + port 2; + + } + + admin-mode enable; + + } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the commands that will be issued in sequence + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data radio xpic create group 2 radio 4 port 1 radio 5 port 2 + radio xpic set group 2 admin enable + } + } + admin@ncs(config-config)# commit + ``` + + + + ### 4.3.11.3 Delete /radio/xpic/group * with admin-mode ENABLED + + - Deleting radio xpic group without admin-mode enabled assumes first disabling the admin mode and only then deleting the radio xpic group. + + - Considering initial config: + ``` + devices device + config + radio xpic group 2 radio 4 port 1 radio 5 port 2 admin-mode enable + ! + ! + ``` + + - We delete the radio xpic group 2: + ``` + devices device + config + no radio xpic group 2 + ! + ! + ``` + + - Commit DRY to display the **diffs from the CDB** candidate state + ``` + admin@ncs(config-config)# commit dry-run + cli { + local-node { + data devices { + device { + config { + radio { + - xpic 2 { + - carrier1 { + - radio 4; + - port 1; + - } + - carrier2 { + - radio 5; + - port 2; + - } + - admin-mode enable; + - } + } + } + } + } + } + } + ``` + + - Commit native to display **device CLI native syntax** of the commands that will be issued: + + ``` + admin@ncs(config-config)# commit dry-run outformat native + native { + device { + name + data radio xpic set group 2 admin disable + radio xpic delete group 2 + } + } + admin@ncs(config-config)# commit + ``` + + - Again, we see a two step approach above, we first must disable admin and only then we can delete the radio xpic group 2. + + +# 5. Built in live-status actions +--------------------------------- + + NONE + + +# 6. Built in live-status show +------------------------------ + + NONE + + +# 7. Limitations +---------------- + + NONE + + +# 8. How to report NED issues and feature requests +-------------------------------------------------- + + **Issues like bugs and errors shall always be reported to the Cisco NSO NED team through + the Cisco Support channel:** + + - + - + + The following information is required for the Cisco NSO NED team to be able + to investigate an issue: + + - A detailed recipe with steps to reproduce the issue. + - A raw trace file generated when the issue is reproduced. + - SSH/TELNET access to a device where the issue can be reproduced by the Cisco NSO NED team. + This typically means both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. + However, it is ok with device access through VPNs, jump servers etc though. + + Do as follows to gather the necessary information needed for your device, here named 'dev-1': + + 1. Enable full debug logging in the NED + + ``` + ncs_cli -C -u admin + admin@ncs# configure + admin@ncs(config)# devices device dev-1 ned-settings ceragon-ip20 logging level debug + admin@ncs(config)# commit + ``` + + 2. Configure the NSO to generate a raw trace file from the NED + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + 3. If the NED already had trace enabled, clear it in order to submit only relevant information + + Do as follows for NSO 6.4 or newer: + + ``` + admin@ncs(config)# devices device dev-1 clear-trace + ``` + + Do as follows for older NSO versions: + + ``` + admin@ncs(config)# devices clear-trace + ``` + + 4. Run a compare-config to populate the trace with initial device config + + ``` + admin@ncs(config)# devices device dev-1 compare-config + ``` + + 5. Reproduce the found issue using ncs_cli or your NSO service. + Write down each necessary step in a reproduction report. + + In addition to this, it helps if you can show how it should work + by manually logging into the device using SSH/TELNET and type + the relevant commands showing a successful operation. + + 6. Gather the reproduction report and a copy of the raw trace file + containing data recorded when the issue happened. + + 7. Contact the Cisco support and request to open a case. Provide the gathered files + together with access details for a device that can be used by the + Cisco NSO NED when investigating the issue. + + + **Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when + applicable. Such requests shall also go through the Cisco support channel.** + + The following information is required for feature requests and extensions: + + 1. Set the config on the real device including all existing dependent config + and run sync-from to show it in the trace. + + 2. Run sync-from # devices device dev-1 sync-from + + 3. Attach the raw trace to the ticket + + 4. List the config you want implemented in the same syntax as shown on the device + + 5. SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification + of the new feature. This usually means that both read and write permissions are required. + Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access + through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET. + + +# 9. How to rebuild a NED +-------------------------- + + To rebuild the NED do as follows: + + ``` + > cd $NED_ROOT_DIR/src + > make clean all + ``` + + When the NED has been successfully rebuilt, it is necessary to reload the package into NSO. + + ``` + admin@ncs# packages reload + ``` + + +# 10. Configure the NED to use ssh multi factor authentication +--------------------------------------------------------------- + + This NED supports multi factor authentication (MFA) using the ssh authentication + method 'keyboard-interactive'. + + Some additional steps are required to enable the MFA support: + + 1. Verify that your NSO version supports MFA. This is configurable as additional + settings in the authentication group used by the device instance. + + Enter a NSO CLI and enter the following and do tab completion: + + ``` + > ncs_cli -C -u admin + admin@ncs# show running-config devices authgroups group default default-map + Possible completions: + action-name The action to call when a notification is received. + callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance. + mfa Settings for handling multi-factor authentication towards the device + public-key Use public-key authentication + remote-name Specify device user name + remote-password Specify the remote password + remote-secondary-password Second password for configuration + same-pass Use the local NCS password as the remote password + same-secondary-password Use the local NCS password as the remote secondary password + same-user Use the local NCS user name as the remote user name + ``` + + If 'mfa' is displayed in the output like above, NSO has MFA support enabled. + In case MFA is not supported it is necessary to upgrade NSO before proceeding. + + 2. Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part + of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the + ssh server and expects to get a proper response in return. + + The executable can be a simple shell script or a program implemented in any programming language. + + The required behaviour is like this: + - read one line from stdin + The line passed from the NED will be a semi colon separated string containing the following info: + ``` + [;;;;;;;] + ``` + The elements for device name, user, password and opaque corresponds to what has been configured in NSO. + The ssh server name, instruction and prompt are given by the ssh server during the authentication step. + + Each individual element in the semi colon separated list is Base64 encoded. + + - Extract the challenge based on the contents above. + + - Print a response matching the challenge to stdout and exit with code 0 + + - In case a matching response can not be given do exit with code 2 + + Below is a simple example of an MFA authenticator implemented in Python3: + + ``` + #!/usr/bin/env python3 + import sys + import base64 + + # This is an example on how to implement an external multi factor authentication handler + # that will be called by the NED upon a ssh 'keyboard-interactive' authentication + # The handler is reading a line from stdin with the following expected format: + # [;;;;;;;] + # All elements are base64 encoded. + + def decode(arg): + return str(base64.b64decode(arg))[2:-1] + + if __name__ == '__main__': + query_challenges = { + "admin@localhost's password: ":'admin', + 'Enter SMS passcode:':'secretSMScode', + 'Press secret key: ':'2' + } + # read line from stdin and trim brackets + line = sys.stdin.readline().strip()[1:-1] + args = line.split(';') + prompt = decode(args[6]) + if prompt in query_challenges.keys(): + print(query_challenges[prompt]) + exit(0) + else: + exit(2) + ``` + + 3. Configure the authentication group used by the device instance to enable MFA. There + are two configurables available: + - executable The path to the external multi factor authentication executable (mandatory). + - opaque Opaque data that will passed as a cookie element to the executable (optional). + + ``` + > ncs_cli -C -u admin + admin@ncs# config + Entering configuration mode terminal + admin@ncs(config)# devices authgroups group default-map mfa executable + admin@ncs(config)# devices authgroups group default-map mfa opaque + admin@ncs(config)# commit + ``` + + 4. Try connecting to the device. + + +## 10.1 Trouble shooting +------------------------ + In case of connection problems the following steps can help for debugging: + + Enable the NED trace in debug level: + + ``` + > devices device dev-1 trace raw + > devices device dev-1 ned-settings ceragon-ip20 logger level debug + > commit + ``` + + Try connect again + + Inspect the generated trace file. + + Verify that the ssh client is using the external authenticator executable: + + ``` + using ssh external mfa executable: + ``` + + Verify that the executable is called with the challenges presented by the ssh server: + + ``` + calling external mfa executable with ssh server given name: '', instruction: '', prompt '' + ``` + + Check for any errors reported by the NED when calling the executable + + ``` + ERROR: external mfa executable failed <....> + ``` diff --git a/checkpoint-gaiaos_rest/README-ned-settings.md b/checkpoint-gaiaos_rest/README-ned-settings.md new file mode 100644 index 00000000..dc8a1ebc --- /dev/null +++ b/checkpoint-gaiaos_rest/README-ned-settings.md @@ -0,0 +1,1413 @@ +# NED settings details +---------------------- + + This NED is equipped with a number of runtime configuration options "NED settings" allowing for + customization by the end user. All options are configurable using the NSO API for NED settings. + Most NED settings can be configured globally, per device profile or per device instance in the + following locations: + + global + /ncs:devices/global-settings/ned-settings/checkpoint-gaiaos_rest/ + profile + /ncs:devices/ncs:profiles/profile:/ned-settings/checkpoint-gaiaos_rest/ + device + /ncs:/device/devices/device:/ned-settings/checkpoint-gaiaos_rest/ + + Profiles setting overrides global-settings and device settings override profile settings, + hence the narrowest scope of the setting is used by the device. + + If user changes a ned-setting, then user must reconnect to the device, i.e. + disconnect and connect in order for the new setting to take effect. + + From the NSO CLI the device instance NED settings for this NED are available under: + + ``` + # config + # devices device dev-1 ned-settings checkpoint-gaiaos_rest + + Press TAB to see all the NED settings. + + ``` + + +# Table of contents +------------------- + + ``` + 1. ned-settings checkpoint-gaiaos_rest + 2. logger + 3. read + 4. domain + 5. config-mode-config + 5.1. block-cli-line + 5.2. exclude-load-config + 5.3. special-prompt-handling + 6. connection + 7. fetch-REST-config + 8. block-nat-rule-positions + 9. block-rest-call-body + 10. functionality-update + 11. transaction-id + 12. rest-show-limits + 13. rest-show-body-params + 14. developer-settings + 14.1. ignore-device-warnings + 14.2. request-body-additions + 15. developer + 16. db-edit-config + 17. vsx-settings + 17.1. vsx-credentials + 18. mgmt + 18.1. object + ``` + + +# 1. ned-settings checkpoint-gaiaos_rest +---------------------------------------- + + + - log-verbose (default false) + + [DEPRECATED] Enabled extra verbose logging in NED (for debugging). + + + - use-mgmt-api (default false) + + Specify if Management REST API configuration should be used. + + + - use-rest-configuration (default true) + + Specify if REST configuration should be used. + + + - use-movable-nat-rules (default false) + + Enable using movable-nat-rules. + + Note: The value of this ned-setting is mirrored in the config YANG model in a hidden leaf and the + value is copied during sync-from. If no sync-from has been done a partial-sync-from on the + hidden leaf needs to be done to avoid failures due to 'when' expressions involving this leaf. + + For example: + admin@ncs(config)# devices partial-sync-from path [ /ncs:devices/device[name='dev-1']/config/use-movable-nat-rule ] + sync-result { + device dev-1 + result true + } + + + - rest-show-limit (default 500) + + [DEPRECATED] Use rest-show-limits. + + + - clear-sessions (default false) + + Specify if stale sessions should be cleared before device write. + + +# 2. ned-settings checkpoint-gaiaos_rest logger +----------------------------------------------- + + Settings for controlling logs generated. + + + - level (default info) + + Set level of logging. + + error - error. + + info - info. + + verbose - verbose. + + debug - debug. + + + - java (default true) + + Toggle wether logs should be added to ncs-java-vm.log. + + +# 3. ned-settings checkpoint-gaiaos_rest read +--------------------------------------------- + + Read from device ned-settings. + + + - parallel-api-calls (default disable) + + Enable/Disable parallel api calls invocation using Threads during sync-from. + + disable - disable. + + enable - enable. + + + - transaction-id-provisional (default true) + + Enable/Disable use of new NSO feature to set provisional transaction-id in show() to save a + call to getTransId() with sync-from. + + + - throw-on-http-fivehundred-codes (default true) + + When a REST API call fails with a 5XX HTTP Status code during sync-from the NED will throw an error instead of just logging it. + Note: Disabling this ned-setting is not recommended as it can lead to config diff issues. + + +# 4. ned-settings checkpoint-gaiaos_rest domain +----------------------------------------------- + + Domain settings for multi-domain. + + + - name + + Specify the domain to log in to. + + + - local-domain-config-only (default false) + + Only store local domain config. + + +# 5. ned-settings checkpoint-gaiaos_rest config-mode-config +----------------------------------------------------------- + + Control the usage of config mode configurations. + + + - use-config-mode-configuration (default false) + + Specify if config mode configuration should be used. + + + - is-gateway-device (default false) + + Specify if this device is a gateway device. + + + - expert-mode-password + + Specify the expert mode password. + + + - save-config (default false) + + Specify if save config should be issued after every transaction. + + + - clienv-rows-all-config (default 0) + + deprecated. + + + - ospf2-instance + + Configuring accept|restrict-all-ipv4 + On the device one can configure an inbound-route-filter with either + "accept-all-ipv4" or "restrict-all-ipv4". In the NED, this is instead + modelled as a boolean, thus accept|restrict-all-ipv4 are configured as + follows: + accept-all-ipv4 -> accept-all-ipv4 true + restrict-all-ipv4 -> accept-all-ipv4 false + + Ospf2 instances + On some devices one can configure inbound-route-filter for different + ospf instances, while on other devices such instances are not used. To + accommodate this, if "instance " is not configurable on your + device, the ned-setting ospf2-instance must be set to some value: + admin@ncs(config)# devices device ned-settings checkpoint-gaiaos_rest config-mode-config ospf2-instance "default" + + Command example for device which support ospf2 instance: + set inbound-route-filter ospf2 instance default rank default + + Command example for device which does not support ospf2 instance: + set inbound-route-filter ospf2 rank default + + + - use-expert-mode (default false) + + Specify if expert mode should be used as standard mode. + + +## 5.1. ned-settings checkpoint-gaiaos_rest config-mode-config block-cli-line +----------------------------------------------------------------------------- + + Specify what config line the NED should not send to the device. + + - block-cli-line + + - name + + Specify what config the NED should not send since the device automatically configures it. + + +## 5.2. ned-settings checkpoint-gaiaos_rest config-mode-config exclude-load-config +---------------------------------------------------------------------------------- + + Exclude config to be loaded into the NED. Only supports the exclusion of + /config_mode_config/rba/role today. + + - exclude-load-config + + - config-component + + Specify the config component to be excluded, for example rba role. + + - config-name + + Specify the config name to be excluded, for example adminRole or a regex + (admin|cloningAdmin|monitor)Role. + + +## 5.3. ned-settings checkpoint-gaiaos_rest config-mode-config special-prompt-handling +-------------------------------------------------------------------------------------- + + Depending on the checkpoint device some commands in config_mode_config might require special handling of the prompt. + + This section lists all commands that the NED have implementation for which require special handling: + set net-access telnet on + If a needed command is not supported then the NED can be enhanced to support it. + + - special-prompt-handling + + - command + + Example: set net-access telnet on. + + +# 6. ned-settings checkpoint-gaiaos_rest connection +--------------------------------------------------- + + Per device connection configuration. + + + - hostname-verification (default true) + + deprecated. + + + - rest-port (default 443) + + Port number for the REST API. + + + - auth enter-last-published-session (default false) + + Login to last published session when you don't need to make any changes or updates. + + + - auth check (default true) + + Check and authenticate on 401 response. + + + - number-of-retries (default 0) + + Configure max number of retries the NED will try to connect to the device before giving up. + Default 0. + + + - time-between-retry (default 1) + + Configure the time in seconds the NED will wait between each connect retry. Default 1s. + + +# 7. ned-settings checkpoint-gaiaos_rest fetch-REST-config +---------------------------------------------------------- + + By default checkpoint-gaiaos_rest NED fetches all the config which the YANG-model supports. + If performance is of importance it is possible to excluding config not needed and thereby increase it. + + Note that the order you set the objects in the NED-settings matter. Set the object with least dependencies first. + For example: Package can contain access-layer, access-layer can contain service-tcp but service-tcp cannot contain any of the other objects. + Therefor we set service-tcp first, then access-layer and finally package. + + - fetch-REST-config + + - name + + Specify what config the NED should fetch. + + +# 8. ned-settings checkpoint-gaiaos_rest block-nat-rule-positions +----------------------------------------------------------------- + + Specify what nat-rule position commands the NED will not send. + + - block-nat-rule-positions + + - name + + Example: 99 would mean that no command involving + nat-rule 99 will be sent + + +# 9. ned-settings checkpoint-gaiaos_rest block-rest-call-body +------------------------------------------------------------- + + Specify the rest path and body to be blocked. + + - block-rest-call-body + + - path + + Specify the rest path to be blocked. Example: set-package. + + - regex + + Regex matching the json body to be blocked. Example: .*installation-targets.*add.*all.*. + + +# 10. ned-settings checkpoint-gaiaos_rest functionality-update +-------------------------------------------------------------- + + Contains flags which updates extisting NED functionality. + + + - access-rule-naming (default false) + + Allows no, unique, or duplicated name for /access-layer/access-rule. + + If enabled then: + * /access-layer/access-rule/name: will only be used as a NED internal identifier + * /access-layer/access-rule/name-on-device: will be sent to the device as rule name if configured + + * If the /access-layer/access-rule was created on the device then: + * if it has a name: + the uid will be used as the value for + /access-layer/access-rule/name + + the name will be used as the value for + /access-layer/access-rule/name-on-device + + * if it does not have a name: + the uid will be used as the value for + /access-layer/access-rule/name + + Note: The value of this ned-setting is mirrored in the config YANG model in a hidden leaf and the + value is copied during sync-from. If no sync-from has been done a partial-sync-from on the + hidden leaf needs to be done to avoid failures due to 'when' expressions involving this leaf. + + For example: + admin@ncs(config)# devices partial-sync-from path [ /ncs:devices/device[name='dev-1']/config/access-rule-naming ] + sync-result { + device dev-1 + result true + } + + +# 11. ned-settings checkpoint-gaiaos_rest transaction-id +-------------------------------------------------------- + + Contains flags which modifies the transaction id calculations. + + + - use-show-changes (default false) + + Specify if show-changes should be used for checking if the ned is in sync. + + + - use-to-session (default true) + + Specify if show-changes should use to-session parameter. + + +# 12. ned-settings checkpoint-gaiaos_rest rest-show-limits +---------------------------------------------------------- + + Specify how many objects each rest call shall fetch. + + + - default (default 500) + + + - access-rulebase (default 150) + + +# 13. ned-settings checkpoint-gaiaos_rest rest-show-body-params +--------------------------------------------------------------- + + + - dereference-group-members (default true) + + Indicates whether to dereference "members" field by details level for every object in reply. + + +# 14. ned-settings checkpoint-gaiaos_rest developer-settings +------------------------------------------------------------ + + Developer settings. + + + - publish (default true) + + Specify if publish shall be issued after every REST commit. + + + - sleep-after-publish (default 0) + + Specify how long in seconds the NED should sleep after mgmt publish. + + + - polling-interval (default 0) + + Configure in seconds how long the NED should wait between show-task calls after publish. + Stops when status is succeeded or progress-percentage is 100 + + + - polling-timeout (default 30) + + Configure the polling timeout(seconds). + + + - send-REST-rollbacks (default false) + + [DEPRECATED]. + + + - cluster-member-interface-filter (default false) + + Enable the filtering of /simple-cluster/cluster-members/interfaces. + Occasionally, error in the device's response has been encountered when set to true. + + The device accepts both the unfiltered and filtered POST for /simple-cluster/cluster-members/interfaces. + + admin@ncs% set devices device ned-settings checkpoint-gaiaos_rest developer-settings + cluster-member-interface-filter true + admin@ncs(config)% commit + admin@ncs(config)% request devices device disconnect + admin@ncs(config)% request devices device sync-from + result true + admin@ncs(config)% + + Note: in order for the above settings to take effect, you must disconnect and do sync-from. + + Test commands: + Step 1: add the cluster + set devices device config simple-cluster SC ipv4-address 1.1.1.22 + set devices device config simple-cluster SC cluster-members M1 interfaces XXX ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + set devices device config simple-cluster SC cluster-members M1 interfaces YYY ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + set devices device config simple-cluster SC cluster-members M2 interfaces XXX ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + set devices device config simple-cluster SC cluster-members M2 interfaces YYY ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + set devices device config simple-cluster SC cluster-members M3 interfaces XXX ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + set devices device config simple-cluster SC cluster-members M3 interfaces YYY ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + commit no-networking + + Step 2: add the interfaces + set devices device config simple-cluster SC interfaces AAA ipv4-mask-length 32 ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 interface-type cluster + set devices device config simple-cluster SC interfaces BBB ipv4-mask-length 32 ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 interface-type cluster + set devices device config simple-cluster SC interfaces CCC ipv4-mask-length 32 ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 interface-type cluster + set devices device config simple-cluster SC cluster-members M1 interfaces AAA ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + set devices device config simple-cluster SC cluster-members M1 interfaces BBB ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + set devices device config simple-cluster SC cluster-members M1 interfaces CCC ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + set devices device config simple-cluster SC cluster-members M2 interfaces AAA ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + set devices device config simple-cluster SC cluster-members M2 interfaces BBB ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + set devices device config simple-cluster SC cluster-members M2 interfaces CCC ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + set devices device config simple-cluster SC cluster-members M3 interfaces AAA ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + set devices device config simple-cluster SC cluster-members M3 interfaces BBB ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + set devices device config simple-cluster SC cluster-members M3 interfaces CCC ipv4-network-mask 255.255.255.0 ipv4-address 1.1.1.11 + commit dry-run outformat native + + Without filtering: + POST /web_api/set-simple-cluster + { + "interfaces": {"add": [{ + "comments": "", + "ipv4-address": "1.1.1.11", + "ipv4-mask-length": 32, + "color": "black", + "interface-type": "cluster", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }]}, + "name": "SC" + } + POST /web_api/set-simple-cluster + { + "interfaces": {"add": [{ + "comments": "", + "ipv4-address": "1.1.1.11", + "ipv4-mask-length": 32, + "color": "black", + "interface-type": "cluster", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }]}, + "name": "SC" + } + POST /web_api/set-simple-cluster + { + "interfaces": {"add": [{ + "comments": "", + "ipv4-address": "1.1.1.11", + "ipv4-mask-length": 32, + "color": "black", + "interface-type": "cluster", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }]}, + "name": "SC" + } + POST /web_api/set-simple-cluster + { + "members": {"update": [ + { + "interfaces": [ + { + "ipv4-address": "1.1.1.11", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "XXX", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "YYY", + "ipv4-network-mask": "255.255.255.0" + } + ], + "name": "M1" + }, + { + "interfaces": [ + { + "ipv4-address": "1.1.1.11", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "XXX", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "YYY", + "ipv4-network-mask": "255.255.255.0" + } + ], + "name": "M2" + }, + { + "interfaces": [ + { + "ipv4-address": "1.1.1.11", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "XXX", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "YYY", + "ipv4-network-mask": "255.255.255.0" + } + ], + "name": "M3" + } + ]}, + "name": "SC" + } + POST /web_api/set-simple-cluster + { + "members": {"update": [ + { + "interfaces": [ + { + "ipv4-address": "1.1.1.11", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "XXX", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "YYY", + "ipv4-network-mask": "255.255.255.0" + } + ], + "name": "M1" + }, + { + "interfaces": [ + { + "ipv4-address": "1.1.1.11", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "XXX", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "YYY", + "ipv4-network-mask": "255.255.255.0" + } + ], + "name": "M2" + }, + { + "interfaces": [ + { + "ipv4-address": "1.1.1.11", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "XXX", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "YYY", + "ipv4-network-mask": "255.255.255.0" + } + ], + "name": "M3" + } + ]}, + "name": "SC" + } + POST /web_api/set-simple-cluster + { + "members": {"update": [ + { + "interfaces": [ + { + "ipv4-address": "1.1.1.11", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "XXX", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "YYY", + "ipv4-network-mask": "255.255.255.0" + } + ], + "name": "M1" + }, + { + "interfaces": [ + { + "ipv4-address": "1.1.1.11", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "XXX", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "YYY", + "ipv4-network-mask": "255.255.255.0" + } + ], + "name": "M2" + }, + { + "interfaces": [ + { + "ipv4-address": "1.1.1.11", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "XXX", + "ipv4-network-mask": "255.255.255.0" + }, + { + "ipv4-address": "1.1.1.11", + "name": "YYY", + "ipv4-network-mask": "255.255.255.0" + } + ], + "name": "M3" + } + ]}, + "name": "SC" + } + + With filtering: + POST https://IP:PORT/web_api/set-simple-cluster + { + "interfaces": {"add": [{ + "comments": "", + "ipv4-address": "1.1.1.11", + "ipv4-mask-length": 32, + "color": "black", + "interface-type": "cluster", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }]}, + "name": "SC" + } + POST https://IP:PORT/web_api/set-simple-cluster + { + "interfaces": {"add": [{ + "comments": "", + "ipv4-address": "1.1.1.11", + "ipv4-mask-length": 32, + "color": "black", + "interface-type": "cluster", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }]}, + "name": "SC" + } + POST https://IP:PORT/web_api/set-simple-cluster + { + "interfaces": {"add": [{ + "comments": "", + "ipv4-address": "1.1.1.11", + "ipv4-mask-length": 32, + "color": "black", + "interface-type": "cluster", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }]}, + "name": "SC" + } + POST https://IP:PORT/web_api/set-simple-cluster + { + "members": {"update": [ + { + "interfaces": [{ + "ipv4-address": "1.1.1.11", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }], + "name": "M1" + }, + { + "interfaces": [{ + "ipv4-address": "1.1.1.11", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }], + "name": "M2" + }, + { + "interfaces": [{ + "ipv4-address": "1.1.1.11", + "name": "AAA", + "ipv4-network-mask": "255.255.255.0" + }], + "name": "M3" + } + ]}, + "name": "SC" + } + POST https://IP:PORT/web_api/set-simple-cluster + { + "members": {"update": [ + { + "interfaces": [{ + "ipv4-address": "1.1.1.11", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }], + "name": "M1" + }, + { + "interfaces": [{ + "ipv4-address": "1.1.1.11", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }], + "name": "M2" + }, + { + "interfaces": [{ + "ipv4-address": "1.1.1.11", + "name": "BBB", + "ipv4-network-mask": "255.255.255.0" + }], + "name": "M3" + } + ]}, + "name": "SC" + } + POST https://IP:PORT/web_api/set-simple-cluster + { + "members": {"update": [ + { + "interfaces": [{ + "ipv4-address": "1.1.1.11", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }], + "name": "M1" + }, + { + "interfaces": [{ + "ipv4-address": "1.1.1.11", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }], + "name": "M2" + }, + { + "interfaces": [{ + "ipv4-address": "1.1.1.11", + "name": "CCC", + "ipv4-network-mask": "255.255.255.0" + }], + "name": "M3" + } + ]}, + "name": "SC" + } + + +## 14.1. ned-settings checkpoint-gaiaos_rest developer-settings ignore-device-warnings +-------------------------------------------------------------------------------------- + + Specify what device CLI errors the NED should ignore. + + - ignore-device-warnings + + - ignore-warning + + Specify what device CLI error the NED should ignore. + + +## 14.2. ned-settings checkpoint-gaiaos_rest developer-settings request-body-additions +-------------------------------------------------------------------------------------- + + Specify what additional key/value the request body shall contain. + + - request-body-additions + + - key + + Specify the key. Example: ignore-warnings. + + - operation + + Specify what operation this should apply to. Example: add. + + - object + + Specify what object this should apply to. Example: service-tcp. + + - value + + Specify the value. Example: true. + + For example, this would ignore warnings and errors when adding service-tcp or service-udp objects + ned-settings checkpoint-gaiaos_rest developer-settings request-body-additions ignore-warnings add service-tcp value true + ned-settings checkpoint-gaiaos_rest developer-settings request-body-additions ignore-warnings add service-udp value true + ned-settings checkpoint-gaiaos_rest developer-settings request-body-additions ignore-errors add service-tcp value true + ned-settings checkpoint-gaiaos_rest developer-settings request-body-additions ignore-errors add service-udp value true + + +# 15. ned-settings checkpoint-gaiaos_rest developer +--------------------------------------------------- + + Contains settings used for debugging (intended for NED developers). + + + - progress-verbosity (default very-verbose) + + Maximum NED verbosity level which will be reported. + + disabled - disabled. + + normal - normal. + + verbose - verbose. + + very-verbose - very-verbose. + + debug - debug. + + + - trace-enable (default false) + + [DEPRECATED] Enable developer tracing. WARNING: may choke NSO with large commits|systems. + + + - platform model + + Override device model name/number. + + + - platform name + + Override device name. + + + - platform version + + Override device version. + + +# 16. ned-settings checkpoint-gaiaos_rest db-edit-config +-------------------------------------------------------- + + Configure the usage of db_edit. + + + - use-db-edit-configuration (default false) + + Specifies whether db_edit configuration should be used. Requires expert password + to be set. + Note that if this is enabled and use-rest-configuration isn't disabled, it could + lead to compare-config diffs since both db_edit and the REST API sometimes + configure the same config. + + +# 17. ned-settings checkpoint-gaiaos_rest vsx-settings +------------------------------------------------------ + + + - use-vsx-provisioning-tool (default false) + + Specify if vsx_provisioning_tool should be used. + + + - auth-group-credentials (default false) + + Specify if the credentials referred to in + devices device DEVICE-NAME authgroup should be used. + + + - trace-vsx-provisioning-tool (default false) + + Specify if vsx_provisiong tool commands which include username/password should be traced. + + +## 17.1. ned-settings checkpoint-gaiaos_rest vsx-settings vsx-credentials +------------------------------------------------------------------------- + + - vsx-credentials + + - ip + + - user (default ) + + - password (default ) + + +# 18. ned-settings checkpoint-gaiaos_rest mgmt +---------------------------------------------- + + + - api address default + + + - api base-url (default /web_api) + + API base URL for device REST API. + + + - api auth domain name + + Use domain to login to specific domain. Domain can be identified by name or UID. + + + - api auth enter-last-published-session (default false) + + Login to the last published session. Such login is done with the Read Only permissions. + + + - api sync enabled + + Objects to be synced. + + host - host. + + network - network. + + address-range - address-range. + + group - group. + + service-tcp - service-tcp. + + service-udp - service-udp. + + service-icmp - service-icmp. + + service-icmp6 - service-icmp6. + + service-sctp - service-sctp. + + service-other - service-other. + + service-dce-rpc - service-dce-rpc. + + service-rpc - service-rpc. + + service-group - service-group. + + access-layer - access-layer. + + access-rule - access-rule. + + application-site-category - application-site-category. + + application-site - application-site. + + domain - domain. + + simple-cluster - simple-cluster. + + simple-gateway - simple-gateway. + + package - package. + + nat-rule - nat-rule. + + + - api sync disabled + + Objects not to be synced. + + host - host. + + network - network. + + address-range - address-range. + + group - group. + + service-tcp - service-tcp. + + service-udp - service-udp. + + service-icmp - service-icmp. + + service-icmp6 - service-icmp6. + + service-sctp - service-sctp. + + service-other - service-other. + + service-dce-rpc - service-dce-rpc. + + service-rpc - service-rpc. + + service-group - service-group. + + access-layer - access-layer. + + access-rule - access-rule. + + application-site-category - application-site-category. + + application-site - application-site. + + domain - domain. + + simple-cluster - simple-cluster. + + simple-gateway - simple-gateway. + + package - package. + + nat-rule - nat-rule. + + + - api config object defaults is-syncable (default true) + + + - api config object defaults request list limit (default 500) + + + - api config object defaults request list parallel-api-calls (default true) + + Enable/Disable parallel api calls invocation. + + + - api config session check-alive (default true) + + + - api config session keep-alive (default true) + + + - conn ssl accept-any (default true) + + Accept any SSL certificate presented by the device. Warning! + This enables Man in the Middle attacks and should only be used + for testing and troubleshooting. + + + - conn ssl certificate + + SSL certificate stored in DER format but since it is entered + as Base64 it is very similar to PEM but without banners like "----- + BEGIN CERTIFICATE -----". Default uses the default trusted certificates + installed in Java JVM. An easy way to get the PEM of a server: + openssl s_client -connect HOST:PORT + + +## 18.1. ned-settings checkpoint-gaiaos_rest mgmt api sync object +----------------------------------------------------------------- + + - api sync object + + - name + + host - host. + + network - network. + + address-range - address-range. + + group - group. + + service-tcp - service-tcp. + + service-udp - service-udp. + + service-icmp - service-icmp. + + service-icmp6 - service-icmp6. + + service-sctp - service-sctp. + + service-other - service-other. + + service-dce-rpc - service-dce-rpc. + + service-rpc - service-rpc. + + service-group - service-group. + + access-layer - access-layer. + + access-rule - access-rule. + + application-site-category - application-site-category. + + application-site - application-site. + + domain - domain. + + simple-cluster - simple-cluster. + + simple-gateway - simple-gateway. + + package - package. + + nat-rule - nat-rule. + + - is-syncable (default true) + + - enabled names + + Included Names during the sync. + + - enabled uids + + Included UIDs during the sync. + + - actions enabled + + LIST - LIST. + + GET - GET. + + CREATE - CREATE. + + UPDATE - UPDATE. + + DELETE - DELETE. + + - actions disabled + + LIST - LIST. + + GET - GET. + + CREATE - CREATE. + + UPDATE - UPDATE. + + DELETE - DELETE. + + - request list limit (default 500) + + - request list parallel-api-calls (default true) + + Enable/Disable parallel api calls invocation. + + - request list filter + + +# 19 Checkpoint modes +===================== +The Checkpoint-gaiaos_rest NED can communicate to the device through these modes: + REST + CLI (called config_mode_config in the NED) + Db_edit + +The Checkpoint-gaiaos_rest NED has NED-settings which affects how the NED will interact with the different modes. +For details, see the referenced ned-settings. + +Summary: + 1.4 ned-setting to use config mode configuration + admin@ncs(config)% set devices device ned-settings + checkpoint-gaiaos_rest config-mode-config use-config-mode-configuration true + + 1.4 ned-setting for using checkpoint-gaiaos_rest against a gateway device + admin@ncs(config)% set devices device ned-settings + checkpoint-gaiaos_rest config-mode-config is-gateway-device true + + 1.13 ned-setting for using db_edit + admin@ncs% set devices device ned-settings + checkpoint-gaiaos_rest db-edit-config use-db-edit-configuration true + + 1 ned-setting for not using the REST API + admin@ncs% set devices device ned-settings + checkpoint-gaiaos_rest use-rest-configuration false + + The above ned-settings will have the following effect on what modes the NED will communicate to the device through: + 1 means that the mode is enabled through the NED-Setting + 0 means that the mode is not affected through the NED-Setting + -1 means that the mode is disabled through the NED-Setting + REST CLI DB_EDIT + 1.4 ... config-mode-config use-config-mode-configuration true 0 1 0 + 1.4 ... config-mode-config is-gateway-device true -1 1 0 + 1.13 ... db-edit-config use-db-edit-configuration true 0 0 1 + 1 ... use-rest-configuration true 1 0 0 + ------------------- + -1 1 1 + + REST CLI DB_EDIT + 1.4 ... config-mode-config use-config-mode-configuration false 0 -1 0 + 1.4 ... config-mode-config is-gateway-device false 1 -1 0 + 1.13 ... db-edit-config use-db-edit-configuration false 0 0 -1 + 1 ... use-rest-configuration false -1 0 0 + -------------------- + -1 -1 -1 + + + If nothing is specified in the NED-Settings these are the standard values: + REST CLI DB_EDIT + 1.4 ... config-mode-config use-config-mode-configuration false 0 -1 0 + 1.4 ... config-mode-config is-gateway-device false 1 -1 0 + 1.13 ... db-edit-config use-db-edit-configuration false 0 0 -1 + 1 ... use-rest-configuration true 1 0 0 + ------------------- + 1 -1 -1 + So by default the NED only communicates through REST mode. + + When the device has more than 500 entries for a REST object, + the NED will by default issue multiple REST calls to get all the objects. diff --git a/checkpoint-gaiaos_rest/README.md b/checkpoint-gaiaos_rest/README.md new file mode 100644 index 00000000..8fac053d --- /dev/null +++ b/checkpoint-gaiaos_rest/README.md @@ -0,0 +1,1765 @@ +# Table of contents +------------------- + + ``` + 1. General + 1.1 Extract the NED package + 1.2 Install the NED package + 1.2.1 Local install + 1.2.2 System install + 1.3 Configure the NED in NSO + 2. Optional debug and trace setup + 3. Dependencies + 4. Sample device configuration + 5. Built in live-status actions + 6. Built in live-status show + 7. Limitations + 8. How to report NED issues and feature requests + 9. How to rebuild a NED + 10. Oper-Data + 11. sync-from verbose and load-native-config file + 12. VSX Provisioning + 13. /packages/movable-nat-rule + 14. Information about the device + 15. Using show-changes for calculating transaction id + 16. Nameless access-section + ``` + + +# 1. General +------------ + + This document describes the checkpoint-gaiaos_rest NED. + + Additional README files bundled with this NED package + ``` + +---------------------------+------------------------------------------------------------------------------+ + | Name | Info | + +---------------------------+------------------------------------------------------------------------------+ + | README-ned-settings.md | Information about all run time settings supported by this NED. | + +---------------------------+------------------------------------------------------------------------------+ + ``` + + Common NED Features + ``` + +---------------------------+-----------+------------------------------------------------------------------+ + | Feature | Supported | Info | + +---------------------------+-----------+------------------------------------------------------------------+ + | netsim | yes | | + | | | | + | check-sync | yes | | + | | | | + | partial-sync-from | yes | | + | | | | + | live-status actions | yes | | + | | | | + | live-status show | no | | + | | | | + | load-native-config | no | | + +---------------------------+-----------+------------------------------------------------------------------+ + ``` + + Verified target systems + ``` + +---------------------------+-----------------+--------+---------------------------------------------------+ + | Model | Version | OS | Info | + +---------------------------+-----------------+--------+---------------------------------------------------+ + | | R80 | Gaia | | + | | | | | + | | R81 | Gaia | | + +---------------------------+-----------------+--------+---------------------------------------------------+ + ``` + + +## 1.1 Extract the NED package +------------------------------ + + It is assumed the NED package `ncs--checkpoint-gaiaos_rest-.signed.bin` has already + been downloaded from software.cisco.com. + + In this instruction the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED package downloaded to: /tmp/ned-package-store + + 1. Extract the NED package and verify its signature: + + ``` + > cd /tmp/ned-package-store + > chmod u+x ncs-6.0-checkpoint-gaiaos_rest-1.0.1.signed.bin + > ./ncs-6.0-checkpoint-gaiaos_rest-1.0.1.signed.bin + ``` + + 2. In case the signature can not be verified (for instance if no internet connection), + do as below instead: + + ``` + > ./ncs-6.0-checkpoint-gaiaos_rest-1.0.1.signed.bin --skip-verification + ``` + + 3. The result of the extraction shall be a tar.gz file with the same name as the .bin file: + + ``` + > ls *.tar.gz + ncs-6.0-checkpoint-gaiaos_rest-1.0.1.tar.gz + ``` + + +## 1.2 Install the NED package +------------------------------ + + There are two alternative ways to install this NED package. + Which one to use depends on how NSO itself is setup. + + In the instructions below the following example settings will be used: + + - NSO version: 6.0 + - NED version: 1.0.1 + - NED download directory: /tmp/ned-package-store + - NSO run time directory: ~/nso-lab-rundir + + A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory: + + ``` + > export NSO_RUNDIR=~/nso-lab-rundir + ``` + + +### 1.2.1 Local install +----------------------- + + This section describes how to install a NED package on a locally installed NSO + (see "NSO Local Install" in the NSO Installation guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Untar the tar.gz file. This creates a new sub-directory named: + `checkpoint-gaiaos_rest-.`: + + ``` + > tar xfz ncs-6.0-checkpoint-gaiaos_rest-1.0.1.tar.gz + > ls -d */ + checkpoint-gaiaos_rest-gen-1.0 + ``` + + 2. Install the NED into NSO, using the ncs-setup tool: + + ``` + > ncs-setup --package checkpoint-gaiaos_rest-gen-1.0 --dest $NSO_RUNDIR + ``` + + 3. Open a NSO CLI session and load the new NED package like below: + + ``` + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package checkpoint-gaiaos_rest-gen-1.0 + result true + } + ``` + + Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like + below instead: + + ``` + > ncs-setup --package ncs-6.0-checkpoint-gaiaos_rest-1.0.1.tar.gz --dest $NSO_RUNDIR + > ncs_cli -C -u admin + admin@ncs# packages reload + reload-result { + package checkpoint-gaiaos_rest-gen-1.0 + result true + } + ``` + + Set the environment variable NED_ROOT_DIR to point at the NSO NED package: + + ``` + > export NED_ROOT_DIR=$NSO_RUNDIR/packages/checkpoint-gaiaos_rest-gen-1.0 + ``` + + +### 1.2.2 System install +------------------------ + + This section describes how to install a NED package on a system installed NSO (see "NSO System + Install" in the NSO Installation Guide). + + It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1. + + 1. Do a NSO backup before installing the new NED package: + + ``` + > $NCS_DIR/bin/ncs-backup + ``` + + 2. Start a NSO CLI session and fetch the NED package: + + ``` + > ncs_cli -C -u admin + admin@ncs# software packages fetch package-from-file \ + /tmp/ned-package-store/ncs-6.0-checkpoint-gaiaos_rest-1.0.tar.gz + admin@ncs# software packages list + package { + name ncs-6.0-checkpoint-gaiaos_rest-1.0.tar.gz + installable + } + ``` + + 3. Install the NED package (add the argument replace-existing if a previous version has been loaded): + + ``` + admin@ncs# software packages install checkpoint-gaiaos_rest-1.0 + admin@ncs# software packages list + package { + name ncs-6.0-checkpoint-gaiaos_rest-1.0.tar.gz + installed + } + ``` + + 4. Load the NED package + + ``` + admin@ncs# packages reload + admin@ncs# software packages list + package { + name ncs-6.0-checkpoint-gaiaos_rest-gen-1.0 + loaded + } + ``` + + +## 1.3 Configure the NED in NSO +------------------------------- + + This section describes the steps for configuring a device instance + using the newly installed NED package. + + - Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + - Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + - Configure a new authentication group (my-group) to be used for this device: + + ``` + admin@ncs(config)# devices authgroup group my-group default-map remote-name \ + remote-password + ``` + + - Configure a new device instance (example: dev-1): + + ``` + admin@ncs(config)# devices device dev-1 address + admin@ncs(config)# devices device dev-1 port + admin@ncs(config)# devices device dev-1 device-type generic ned-id checkpoint-gaiaos_rest-gen-1.0 + admin@ncs(config)# devices device dev-1 state admin-state unlocked + admin@ncs(config)# devices device dev-1 authgroup my-group + ``` + - Finally commit the configuration + + ``` + admin@ncs(config)# commit + ``` + + - Verify configuration, using a sync-from. + + ``` + admin@ncs(config)# devices device dev-1 sync-from + result true + ``` + + If the sync-from was not successful, check the NED configuration again. + + +# 2. Optional debug and trace setup +----------------------------------- + + It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting) + + This can be achieved by configuring NSO to generate a trace file for the NED. A trace file + contains information about all interactions with the device. Messages sent and received as well + as debug printouts, depending on the log level configured. + + NSO creates one separate trace file for each device instance with tracing enabled. + Stored in the following location: + + `$NSO_RUNDIR/logs/ned-checkpoint-gaiaos_rest-gen-1.0-.trace` + + Do as follows to enable tracing in one specific device instance in NSO: + + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable trace raw: + + ``` + admin@ncs(config)# devices device dev-1 trace raw + admin@ncs(config)# commit + ``` + + Alternatively, tracing can be enabled globally affecting all configured device instances: + + ``` + admin@ncs(config)# devices global-settings trace raw + admin@ncs(config)# commit + ``` + + 4. Configure the log level for printouts to the trace file: + + ``` + admin@ncs(config)# devices device dev-1 ned-settings checkpoint-gaiaos_rest logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + Alternatively the log level can be set globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices device global-settings ned-settings checkpoint-gaiaos_rest logger \ + level [debug | verbose | info | error] + admin@ncs(config)# commit + ``` + + The log level 'info' is used by default and the 'debug' level is the most verbose. + + **IMPORTANT**: + Tracing shall be used with caution. This feature does increase the number of IPC messages sent + between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should + normally be disabled in production systems. + + + An alternative method for generating printouts from the NED is to enable the Java logging mechanism. + This makes the NED print log messages to common NSO Java log file. + + `$NSO_RUNDIR/logs/ncs-java-vm.log` + + Do as follows to enable Java logging in the NED + + 1. Start a NSO CLI session: + + ``` + > ncs_cli -C -u admin + ``` + + 2. Enter configuration mode: + + ``` + admin@ncs# configure + Entering configuration mode terminal + admin@ncs(config)# + ``` + + 3. Enable Java logging with level all from the NED package: + + ``` + admin@ncs(config)# java-vm java-logging logger com.tailf.packages.ned.checkpointgaiaosrest \ + level level-all + admin@ncs(config)# commit + ``` + + 4. Configure the NED to log to the Java logger + + ``` + admin@ncs(config)# devices device dev-1 ned-settings checkpoint-gaiaos_rest logger java true + admin@ncs(config)# commit + ``` + + Alternatively Java logging can be enabled globally affecting all configured + device instances using this NED package. + + ``` + admin@ncs(config)# devices global-settings ned-settings checkpoint-gaiaos_rest logger java true + admin@ncs(config)# commit + ``` + + **IMPORTANT**: + Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not + affected. However, all log printouts from all log enabled devices are saved in one single file. + This means that the usability is limited. Typically single device use cases etc. + + +# 3. Dependencies +----------------- + + This NED has the following host environment dependencies: + + - Java 1.8 (NSO version < 6.2) + - Java 17 (NSO version >= 6.2) + - Gnu Sed + + Dependencies for NED recompile: + + - Apache Ant + - Bash + - Gnu Sort + - Gnu awk + - Grep + - Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob) + + +# 4. Sample device configuration +-------------------------------- + + NONE + + +# 5. Built in live-status actions +--------------------------------- + + 5.1 Install policy + % request devices device live-status exec install-policy args { access true/false policy-package targets [ ] } + + 5.2 Run script + % request devices device live-status exec run-script args { script