<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vaughn Marshall on Medium]]></title>
        <description><![CDATA[Stories by Vaughn Marshall on Medium]]></description>
        <link>https://medium.com/@vaughn.marshall?source=rss-c3b69c22bfff------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ZA5e9KoleWsSNzoBg7_2gw.png</url>
            <title>Stories by Vaughn Marshall on Medium</title>
            <link>https://medium.com/@vaughn.marshall?source=rss-c3b69c22bfff------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 16:27:14 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vaughn.marshall/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Centrally Managing Scripts with Endevor Team Build]]></title>
            <link>https://medium.com/modern-mainframe/centrally-managing-scripts-with-endevor-team-build-ec6a9b961ed4?source=rss-c3b69c22bfff------2</link>
            <guid isPermaLink="false">https://medium.com/p/ec6a9b961ed4</guid>
            <category><![CDATA[endevor]]></category>
            <category><![CDATA[build-automation]]></category>
            <category><![CDATA[cobol]]></category>
            <category><![CDATA[mainframe]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Vaughn Marshall]]></dc:creator>
            <pubDate>Thu, 10 Nov 2022 15:39:34 GMT</pubDate>
            <atom:updated>2024-06-03T15:19:43.226Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Links of a chain, interwoven with VS Code showing Git submodules, illustrating how submodules can be used to centrally manage Team Build scripts" src="https://cdn-images-1.medium.com/max/1024/1*kohY1giXgmD-uXmCVHlqXQ.png" /><figcaption>Managing scripts with linked Git submodules</figcaption></figure><h3>Team Build Isn’t Just for Developers</h3><p>For customers considering orchestrating and governing their mainframe development natively with their Enterprise DevOps tools stack, Broadcom’s <a href="https://medium.com/modern-mainframe/introducing-ca-endevor-team-build-dab73251040f">Team Build</a> is a key enabler. It allows mainframe builds to be done regardless of where the source code is housed by decoupling the build engine from the traditional all-in-one Endevor ecosystem. Very often in off-host development, build scripts are maintained by the team and versioned with the code in their Git repository. The continuous integration process simply fetches the script along with the code and uses it with the build engine to produce the binaries. With Team Build, things are no different and it certainly supports this way of working. For some though, this presents a challenge: mainframe expertise for builds has long been centralized; many developers are not fully versed in what can be very complex build logic and have relied on tools like Endevor to provide this for them. The good news is that with Team Build, build scripts can still be centralized if so desired.</p><h3>Dedicated Script Repositories</h3><p>With traditional Endevor, processors are managed by Endevor itself. When you invoke a Generate on an element, the assigned processor group determines the build script that gets used and the actual file / processor that provides the logic is pulled from a controlled location in Endevor. Typically, only certain users have access to update the logic but it is usable by all developers. In a native Git implementation, you can opt to manage things the exact same way: by creating a dedicated repository for the build scripts.</p><p>The logic stored in Endevor, once initialized as Team Build scripts can be checked into a central build scripts repository instead of the application source repository. Maintaining scripts in a version control system instead of an external location like a database or controlled PDS makes a ton of sense and is why processors are managed by Endevor itself. For one, you can apply the same discipline for updating the scripts that you would any other application source. You can control access to update, you can branch and have different versions available and you can use things like pull requests to review and approve changes. If your Endevor Admin (who typically managed build logic for their teams) prefers working in Endevor, you can even maintain the Team Build scripts in traditional Endevor and publish them to a central repo with <a href="https://medium.com/modern-mainframe/devops-dozen-award-winner-ca-endevor-bridge-for-git-c4a8754401ba">Bridge for Git</a>.</p><h3>Working with Central Script Repositories</h3><p>When scripts are centralized, developers are going to need to ensure they can access them whenever builds are needed. This can be done by one of two methods. The first way is to simply fetch the central repository separately and refer to it from your scripts. In this scenario, your project directory might look something like this:</p><figure><img alt="VS Code showing a parent folder opened with two repositories — one application repository, and the other a build script repository" src="https://cdn-images-1.medium.com/max/1024/0*9MwX505sXpV9Ceo4" /><figcaption>Application repository and scripts repository cloned into shared parent directory</figcaption></figure><p>The two repositories are cloned under a top level folder and by referencing the scripts folder in your build scripts it’s just a matter of ensuring you have both on your machine.</p><p>There’s an easier way to do this though by using something called a <a href="https://git-scm.com/book/en/v2/Git-Tools-Submodules">Git submodule</a>. Using submodules, you can embed a reference to the other repository in your application repository so that whenever someone makes a clone of it, the repository containing the scripts is also cloned.</p><figure><img alt="Scripts repository embedded as a submodule in application repository" src="https://cdn-images-1.medium.com/proxy/1*mLTtp4yEqpJ4dM8OMP0yNA.png" /><figcaption>Scripts repository embedded as a submodule in application repository</figcaption></figure><p>Notice in the above screenshot that the scripts is located under the application repository and there is a file in the folder .gitmodules that specifies the link to the scripts repository. VS Code also highlights the folder in blue to show it is a submodule providing the files.</p><p>Creating a submodule reference is simple with the following command:</p><pre>git submodule add <em>&lt;remote_url&gt; &lt;destination_folder&gt;</em></pre><p>Once you’ve created the link, you commit and push the change like any other. When cloning the project though, make sure you include the <br>--recurse-submodules flag to get the files from the submodule repository:</p><pre>git clone --recurse-submodules <em>&lt;remote_url&gt;</em></pre><h3>Submodule Versioning</h3><p>Submodules link to a specific version of a repository. This may be a good or a bad thing depending on your use case. You may wish to control which version of a set of scripts is being used to keep things stable, especially while changes are still being worked on. However, if you prefer to always use the latest version of a repository, create a wrapper build script that issues the git submodule update — remote command before calling <a href="https://medium.com/modern-mainframe/a-new-build-tool-that-mainframe-software-developers-cannot-miss-af6b0f5cfa">Team Build’s </a><a href="https://medium.com/modern-mainframe/a-new-build-tool-that-mainframe-software-developers-cannot-miss-af6b0f5cfa">bldz</a> command to refresh to the latest version of the scripts in the centralized script repository. Alternatively, you can embed the command into the Team Build script itself so it happens with every run of bldz. Note though that if you want to update the reference in the application repository to the latest version of the scripts repository for future runs, the git submodule update command will not be enough. After you run the update, you need to commit and push the updated reference in the application repository just like you would any other source code artifact.</p><h3>Summary</h3><p>Submodules provide a convenient way to include other parts of your application that might be managed by other teams into your own application’s repository. Today we talked about build scripts but it could just as easily be a repository of centrally managed copybooks. The copybooks could even come from a team that maintains them from a traditional 3270 Endevor interface via a <a href="https://medium.com/modern-mainframe/devops-dozen-award-winner-ca-endevor-bridge-for-git-c4a8754401ba">Bridge for Git</a> managed copy. Either way, applying source code management techniques to manage your application build scripts, whether the scripts are managed in Endevor or in Git, should be considered a best practice.</p><p>If you’ve found this blog useful and would like to learn more about modernizing your mainframe development practices, you can do so at our <a href="https://medium.com/modern-mainframe">Modern Mainframe</a> blog site where we talk about integrating mainframe into Enterprise tool chains with practical examples and use cases.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ec6a9b961ed4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/modern-mainframe/centrally-managing-scripts-with-endevor-team-build-ec6a9b961ed4">Centrally Managing Scripts with Endevor Team Build</a> was originally published in <a href="https://medium.com/modern-mainframe">Modern Mainframe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Self-service Developer Workspaces in Endevor]]></title>
            <link>https://medium.com/modern-mainframe/self-service-developer-workspaces-in-endevor-3b83c72bdc14?source=rss-c3b69c22bfff------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b83c72bdc14</guid>
            <category><![CDATA[git]]></category>
            <category><![CDATA[agile]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[mainframe]]></category>
            <category><![CDATA[endevor]]></category>
            <dc:creator><![CDATA[Vaughn Marshall]]></dc:creator>
            <pubDate>Tue, 05 Apr 2022 11:20:39 GMT</pubDate>
            <atom:updated>2022-04-05T11:20:39.177Z</atom:updated>
            <content:encoded><![CDATA[<h3>Supports Agile Development Principles like Developer Isolation</h3><p>Agile has become the methodology of choice for software development and for good reasons. Organizations compete in a marketplace driven by regular disruption so improving developer velocity is not only an advantage, it’s critical. To achieve maximum velocity, there are a number of key development practices that must be observed — ensuring developers can make their changes in isolation from other developers to avoid creating unnecessary dependencies is one of those.</p><p>In the world of distributed systems development, this is done by using branches to isolate your code and leveraging virtualization and containers to ensure changes can be built, deployed, and tested without impacting other developers’ work. But how does this translate to the world of mainframe development, especially where veteran mainframers prefer to work with traditional mainframe SCMs instead of Enterprise Git platforms?</p><h3>Taking Inspiration from the Past</h3><p>Parallel development in the world of mainframe is nothing new — organizations typically create parallel lifecycle paths in Endevor for this very purpose. They will have, for example, one or more parallel Endevor environments — DEV1, DEV2, DEV3, and so on — to allow developers to pick a place to start developing with the parallel changes merging at some point later in the life cycle. What <em>is</em> new is the practice of doing this for every separate feature or fix to ensure changes don’t get intertwined unnecessarily.</p><p>At Broadcom, we ran into that very problem early in our own Agile transformation so we created Endevor “sandboxes” — which use the application inventory classification field named “Subsystem” to act as a branch of sorts. Unfortunately, for many customers, this doesn’t work because it requires adjustment of all their build and lifecycle automation, an effort many administrators do not have the cycles to undertake. Additionally, not everyone’s application changes fit neatly into a single subsystem/sandbox. So we needed to come up with something better.</p><p>One way to make this more widely adoptable would be to take inspiration from how Endevor developers have always done parallel development — with parallel environments. Easier said than done! Setting up new environments for every feature or fix would create a huge administrative burden, not to mention being a great way to chew up a ton of unnecessary disk space.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aw2nlFZmqWzXExPM4Ri47w.jpeg" /><figcaption>Dynamic Environments in Endevor</figcaption></figure><h3>Introducing Dynamic Environments and Deferred File Creation</h3><p>Because developer isolation is so critical, we addressed those issues so that developers could isolate their changes regardless of how they choose to work. <a href="https://techdocs.broadcom.com/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-1/administrating/enable-dynamic-environments.html">Dynamic Environments</a> lets developers work with their own environment in Endevor whether they are working in ISPF or if they are using the environment to back their <a href="https://medium.com/modern-mainframe/editing-synchronized-endevor-elements-locally-ff096d09eb5e">Zowe workspace</a> or build their <a href="https://docs.broadcom.com/doc/endevor-software-change-manager-bridging-enterprise-git">Git working directory</a>.</p><p>To ensure that creating these environments did not become an administrative burden, we introduced new SCL that allowed you to define an existing environment like another reference environment, and a way to specify the backing <a href="https://techdocs.broadcom.com/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-1/administrating/enable-deferred-file-creation.html">datasets so they are only allocated on first use</a>, thereby avoiding the unnecessary use of disk space that would be required by a full environment otherwise.</p><h3>Getting Started</h3><p>Enabling Dynamic Environments is really a 2-step process — enabling Dynamic Environments and then setting up Deferred File Creation to ensure the new environments don’t use unnecessary disk space. To enable Dynamic Environments, you must create a Global Control File (GCF) and then add a new parameter to the TYPE=MAIN macro in the C1DEFLTS defaults table. The GCF is a VSAM file that contains the environment definitions and related records for ACM. There is a sample JCL file for allocating the GCF contained in the CSIQJCL member BC1JJB08. The exact steps can also be <a href="https://techdocs.broadcom.com/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-1/administrating/enable-dynamic-environments.html">found here</a>.</p><p>To enable the Deferred File Creation feature, a member must be added to your PARMLIB data set that defines the parameters necessary to allocate files as new when they are being allocated for the first time. The Deferred File Creation member contains statements that will be parsed at element action execution time and used during file allocations if a file does not exist. You can use Endevor symbols to define your data set names within this member as well. Once you’ve added the new member, you need to define the name of the Deferred File Creation member to your C1DEFLTS table TYPE=MAIN section using the DFCMBR= parameter and then reassemble C1DEFLTS. The exact steps can also be <a href="https://techdocs.broadcom.com/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-1/administrating/enable-deferred-file-creation.html">found here</a>.</p><p>One thing to keep in mind with these new environments is security. Access to define or modify environment information is controlled at the environment level by the “PRIMARY_OPTIONS” entry in the security table, BC1TNEQU. When MENUITEM is included in the PRIMARY_OPTIONS entry it resolves to a value for the primary option selected at run time, which for the ENVIRONMENT option is “ENVRMENT”. For example, in the delivered version of BC1TNEQU, we have this entry for PRIMARY_OPTIONS:</p><pre>NAMEQU PRIMARY_OPTIONS, +<br>      L1=(‘C1’), +<br>      L2=(ENVIRONMENT), +<br>      L3=(‘PMENU’), +<br>      L4=(MENUITEM)</pre><p>A request to define dynamic environment “WRK1 ‘’ would result in this pseudo data set name being checked for Read access: C1.WRK1.PMENU.ENVRMENT. If the user has read authority to C1.WRK1.PMENU.ENVRMENT, they will be able to define, update or delete the WRK1 Dynamic Environment, as well as the Systems, Subsystems, etc. within the environment. Note: Static environments defined in C1DEFLTS cannot be updated in this way, they must continue to be updated via changes to C1DEFLTS.</p><p>In order to avoid having to do continual security maintenance and facilitate self-service, having an environment naming convention is key. You may want to consider using a prefix or range of values that can be used for defining dynamic environments and creating a profile or standard access that can be given to a group of users for this purpose. For example, granting read access to this group for data set “C1.W%%%.PMENU.ENVRMENT” would enable them to define dynamic environments that begin with the letter W followed by 3 additional characters. Or you could create permissions for a set group of values to be used; WRK1, WRK2, WRK3, etc.</p><p>We recommend you incorporate the Dynamic environment name into all data sets and definitions that pertain to it. This includes the VSAM MCF files for the environment as well as the Base, Delta, Type, Source Output and Processor output files. This should make it easy to identify all of the data sets that belong to a dynamic environment and to establish rules to secure these data sets. For example, the base library for a type definition could be something like this, including the system, subsystem, environment, stage number and type in the data set name:</p><p>NDV.&amp;C1SY..&amp;C1SU..&amp;C1EN..S&amp;C1S#..&amp;C1ELTYPE</p><p>More details can be found <a href="https://techdocs.broadcom.com/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-1/administrating/enable-dynamic-environments/dynamic-environments-and-security.html">here</a>.</p><h3>Enabling Developers to Serve Themselves</h3><p>To truly enable developers to go fast, you should consider making the creation of a new environment something developers can do themselves. There are a few approaches. If you are looking for an Endevor ISPF-native solution, you can enable the self-service creation environment via processor automation. You can, for example, have a type <em>DYNENV</em> which holds a simple text element defining the name of the requested environment and any input parameters needed for the automation such as the reference environment to create the new one as a clone. Invoking generate could then spawn the actual automation that creates the environment. Here is an <a href="https://github.com/BroadcomMFD/broadcom-product-scripts/blob/main/endevor/Self-servicing-Project-Workareas-in-Endevor-with-Dynamic-Environments/GDYNENV.PROCESS">example of what I mean</a>.</p><p>When the SCL runs, definitions are copied as-is from one environment/system to another. One consideration is the fact that Endevor cannot automatically update symbolics; these are copied as part of cloning of TYPE definitions. If changes have to be made, that would have to be done after the cloning. It can also be done as part of the processor that creates a DYNENV element by giving the element content and using that content in the processor.</p><p>If you want even more control, such as including approvals by the Endevor admin, or would like to tie a form to the entire process you might also consider creating a front end for the process in a Service Catalog solution or a CI orchestrator like Jenkins. In this case, you’d be invoking the same SCL but you would submit it using Zowe CLI’s zowe endevor submit scl.</p><p>Using the Move processor with packages tied to approvals is another Endevor-native way you could add approvals to the mix.</p><h3>What about Cleanup?</h3><p>As with branches in Enterprise Git repositories, you probably don’t want all these project work areas to stick around forever so having a plan to clean up when development is done is recommended. If you want an Endevor ISPF-native solution, you can simply have a delete processor set up to run when the <em>DYNENV</em> elements described above are deleted. <a href="https://github.com/BroadcomMFD/broadcom-product-scripts/blob/main/endevor/Self-servicing-Project-Workareas-in-Endevor-with-Dynamic-Environments/DDYNENV.PROCESS">Here is an example of that</a>.</p><p>Note: any environment containing elements that have not been promoted will not be allowed to be deleted (for good reason!) As an administrator, you can leverage the CSV utility (as well as Zowe CLI) to list out environments and find which ones are dynamic (via the “Dynamic” flag in the output). From there a simple List element CSV or Zowe CLI call will reveal if there are any elements left in the environment. If there are, you can look at the last update date. If that date ages out you can take action to follow up with the developer to clean up.</p><h3>Taking it even further</h3><p>Dynamic environments provide a project work area in native Endevor, accessible via ISPF or Quick Edit but in some cases, developers may want to access it via a branch in Git. If you are using an orchestrator to drive the process, consider using the new Bridge for Git Zowe CLI command that allows you to do just that as part of the overall self-service request:</p><pre>zowe ebg branch add <em>MY-CONTEXT MY-MAPPING MY-BRANCH</em> -like <em>TEMPLATE_BRANCH</em> -endevor-environment <em>DIFFERENT_ENVIRONMENT</em></pre><p>Here’s an example using parameters:</p><pre>zowe ebg branch add myorg dyn-env-repo-name de01 -like dev -endevor-environment DE01</pre><p>So in the above example, I’m adding a new mapping to a Git repo in “myorg” called “dyn-env-repo-name” to a new branch called “de01” which maps the same systems and subsystems in the “dev” branch to the newly created dynamic environment called “DE01”.</p><p>You can even do any other environmental preparation activities you want to go with this. Things like spinning up associated CICS regions or zD&amp;T instances, populating with test data, etc. If you are triggering this from a processor, a webhook can be used to layer an additional workflow that needs to run from off-host.</p><h3>Summary</h3><p>With Dynamic Environments &amp; Deferred File Creation, Endevor developers can create the isolation so critical to achieving the velocity necessary to meet their customer’s demands in a highly competitive environment. At a high level:</p><ul><li>Dynamic environments support Agile by allowing developers to work in isolation and in parallel <strong>regardless of how developers choose to work</strong></li><li>Creating environments is a much lighter lift administratively</li><li>Unnecessary disk usage is avoided with deferred file creation</li><li>Add self-service to allow developers to create environments as needed</li><li>Regularly clean up old environments automatically</li></ul><p>Learn more about <a href="https://www.broadcom.com/products/mainframe/product-portfolio/open-source-mainframe-zowe">Zowe at this site</a>; more <a href="https://medium.com/zowe">Zowe blogs here</a>.</p><p>If you have any questions or feedback or would like to talk about any of your Endevor use cases, please feel free to contact me at <a href="/service/mailto:Vaughn.Marshall@broadcom.com">Vaughn.Marshall@broadcom.com</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b83c72bdc14" width="1" height="1" alt=""><hr><p><a href="https://medium.com/modern-mainframe/self-service-developer-workspaces-in-endevor-3b83c72bdc14">Self-service Developer Workspaces in Endevor</a> was originally published in <a href="https://medium.com/modern-mainframe">Modern Mainframe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Observability for CA Endevor with the Elastic (ELK) Stack]]></title>
            <link>https://medium.com/modern-mainframe/observability-for-ca-endevor-with-the-elastic-elk-stack-d76556864773?source=rss-c3b69c22bfff------2</link>
            <guid isPermaLink="false">https://medium.com/p/d76556864773</guid>
            <category><![CDATA[elastic]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[mainframe]]></category>
            <category><![CDATA[endevor]]></category>
            <dc:creator><![CDATA[Vaughn Marshall]]></dc:creator>
            <pubDate>Thu, 15 Jul 2021 12:17:39 GMT</pubDate>
            <atom:updated>2024-06-03T16:03:41.895Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>What is “Observability”?</strong></p><p>The term observability has been gaining traction with the rise of Site Reliability Engineering as a practice but what does it mean exactly? Observability refers to the ability to infer the internal state of a system by examining its outputs. In the context of software systems, that often means log and event data, runtime metrics like resource usage, and so on. Typically, observability platforms provide you with the ability to transform, aggregate and store these outputs so you can perform analytics on them.</p><p><strong>What are the benefits of doing this?</strong></p><p>By applying observability to a system, you can look for patterns or indicators of problems in the system. You can also establish baseline behaviors to better understand what constitutes normal behavior and what is an anomaly. If you do see problems, you can easily search the aggregated data to get indicators that will help you track down the root cause of problems. Let’s look at a concrete example. Your shop is trying to control their MIPS consumption and you’d like to know if any spikes in developer build activity occur. By adding observability to a system like CA Endevor, you can detect this and alert on threshold breaches in order to see if there is a problem. If you see a pattern of repeated spikes in the number of builds, you may decide that you want to limit foreground execution and control the number of concurrent builds that can be executed to better control MIPS usage.</p><figure><img alt="Sample Endevor Dashboard in Kibana" src="https://cdn-images-1.medium.com/max/1024/1*lgBt4fOTKvXf7X74DVJQGQ.png" /><figcaption>Sample Endevor Dashboard in Kibana</figcaption></figure><p><strong>How can I add observability to CA Endevor?</strong></p><p>To get started, you will need to choose a platform. The guide below will use the Elastic stack, sometimes referred to as the ELK stack. ELK stands for (E)lastic (L)ogstash (K)ibana, which are all components needed for the platform. Elastic is a database where events and data can be aggregated. Logstash is a tool for collecting system outputs, optionally transforming them and then storing them in the Elastic database. Finally, Kibana is a platform for building visualizations and dashboards from the data. There are other platforms and systems out there such as Splunk but the ELK stack is one that is open source (with commercial options available) and widely used.</p><p>Once you have your platform, you need to set it up to gather the data from CA Endevor. There are a number of ways you can do this and you may want to gather multiple outputs depending on your use case. If you are collecting CA Endevor SMF records, you can gather them using Zowe CLI and Logstash. <a href="https://www.ibm.com/docs/en/zos/2.3.0?topic=programs-using-ifasmfdp-smf-data-set-dump-program">Using IFASMFDP</a> and Zowe’s batch commands, you can dump SMF log data and then fetch the outputs to feed into Logstash. Another option might be to scrape data from the Endevor REST API. For this introduction, we are going to use webhooks to capture all the events happening in Endevor in real time via the Logstash http input plugin.</p><p><strong>Getting started</strong></p><p>The first step will be to deploy the ELK stack. As with any enterprise grade system, installing and configuring software such as the ELK stack can be very involved. For the purpose of this introductory article, however, we will focus only on getting the various components installed with minimal configuration so that you can play and get inspired as to how you might integrate Endevor to your organization’s observability platform. You may also find it easier to opt for a <a href="https://www.elastic.co/observability">trial hosted version at elastic.co</a> or to leverage a <a href="https://www.elastic.co/guide/en/elastic-stack-get-started/master/get-started-docker.html">containerized instance</a>.</p><p>Hint: ensure you will have no firewalls or other network issues between your Endevor instance and the ELK install. This likely rules out using laptops — you will want to set this up on a virtual or other server behind your firewall that you have access to.</p><p>To begin, you must download all the software from the <a href="https://www.elastic.co/downloads/">Elastic site</a>. You need to download Elasticsearch, Kibana and Logstash. Once downloaded, I created a parent directory and simply extracted each component into their own directory under the parent.</p><p>In each component directory, you will see a “bin” folder containing scripts to run and perform other operations. The main script to start the component is simply the script named after the component (so for instance for Kibana, you will find bin/kibana.bat for Windows or bin/kibana for Linux installations).</p><p>If you want to change ELK component ports or network bindings, you can edit the *.yml for the component name in the “config” or “conf” subdirectories. For my install, I only needed to update the Kibana.yml so that the server host was the DNS name or IP of the server on which I installed all the components instead of the default “localhost” so that it would be accessible from outside the machine:</p><pre># Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.</pre><pre># The default is ‘localhost’, which usually means remote machines will not be able to connect.</pre><pre># To allow connections from remote users, set this parameter to a non-loopback address.</pre><pre>server.host: “my.server.fully.qualified.host.name”</pre><p>Apart from that, I simply started Elasticsearch using the script, followed by Kibana and verified I could connect using the Kibana URL: <a href="http://my.server.fully.qualified.host.name:5601/">http://<em>my.server.fully.qualified.host.name</em>:5601/</a></p><p>Logstash requires a bit more configuration. In particular, Logstash uses various plugins, filters and other components to capture data from your system. You need to specify in your config/logstash.conf file which input plugins are active, which filters you want (if any) and where to output the results. The configuration file I used looks like this:</p><pre># Sample Logstash configuration <br>input {<br>  http {<br>    host =&gt; &quot;0.0.0.0&quot;<br>    port =&gt; 31311<br>  }<br>}<br>filter {<br>    mutate {<br>        rename =&gt; { &quot;actiondate&quot; =&gt; &quot;date&quot; }<br>        rename =&gt; { &quot;actiontime&quot; =&gt; &quot;time&quot; }<br>    }<br>    truncate{<br>        fields =&gt; &quot;time&quot;          <br>        length_bytes =&gt; 6<br>    }       <br>    mutate {<br>        add_field =&gt; { &quot;eventdatetime&quot; =&gt; &quot;%{date}%{time}&quot; }<br>    }<br>    date {<br>        match =&gt; [ &quot;eventdatetime&quot;, &quot;yyyyMMddHHmmss&quot; ]<br>    }<br>}<br>output {<br>  elasticsearch {<br>    hosts =&gt; [&quot;<a href="http://localhost:9200/">http://localhost:9200</a>&quot;]<br>  }<br>}</pre><p>Let’s break it down to understand what this does. The first section, “input”, indicates what plugins you want to gather input from. In my case, I set up the <a href="https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http.html">http input plugin</a> which sets up a listener on the specified port and address on the logstash server (0.0.0.0 means bind to any NIC on the server). This listener is going to be the end point I post webhooks to.</p><p>The next section, “filter”, describes the filters I want applied to transform the incoming webhook data. I use the mutate plugin to rename and standardize the date/time and actiondate/actiontime on the package and element events. I also shave off the milliseconds part of the event time using the truncate plugin. I then concatenate the event date &amp; time to a new field called eventdatetime using mutate and finally use the date plugin to set the new field to the @timestamp field for the captured data. @timestamp is a standard ELK field that indicates the date/time an event occurs.</p><p>The final section, “output”, says to take the transformed captured webhook data and store it in the elasticsearch database at the specified host and port. In this case it’s on the same machine and I’m using the default port of 9200.</p><p>You should be able to cut &amp; paste the above to a config/logstash.conf file which you will need to create. Afterwards, save your config/logstash.conf file and then start logstash using the bin/logstash script as follows (Note: this assumes you are in the bin directory where the startup scripts are):</p><pre>logstash -f ../config/logstash.conf</pre><p>At this point, you are ready to start sending CA Endevor data.</p><p><strong>Setting up CA Endevor Webhooks</strong></p><p>There are two main steps for setting up CA Endevor to send webhook data. The first is to define the event logging to a USS directory. Logs can be organized by event type, system, etc. and once that is defined, then as events happen in CA Endevor, small files containing event data that will serve as webhook payloads will be written to the file system. To set up Endevor to log events, create the PARMLIB member for the SCL statements. Within the PARMLIB member, code one Package Logging statement, or one Element Logging statement, or both. For example:</p><pre>PACKage LOGging ON<br>  TO PATH<br>    u/users/endevor/user5/pkglog .</pre><pre>ELEment LOGging ON<br>  TO PATh<br>    /u/users/endevor/user08/elmlog<br>  FROm<br>    ENV QA .</pre><p>Of course, you may want to adjust to filter logging for only certain systems or events and if so you need to adjust the SCL. A detailed description on how to set this up can be found <a href="https://techdocs.broadcom.com/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-1/installing/activity-logging-for-webhook-support.html">here in the CA Endevor documentation under the “Activity Logging for Webhook Support” topic</a>.</p><p>Note: if you are not the Endevor administrator, you will likely need to enlist them to set this up. It may also already be set up in which case you simply need to ask them to provide the directories where events are being logged.</p><p>With the PARMLIB member set up, you next need to specify the name of the PARMLIB member as the value for the C1DEFLTS parameter, E2ELOGMBR. You should also create the USS directories associated with the SCL to store the log files (in the example, this is u/users/endevor/user5/pkglog and /u/users/endevor/user08/elmlog). Ensure that the Alternate ID is enabled and can write to the USS directories. If the Alternate ID is enabled, then the log files are written under the security context of the Alternate ID. If the Alternate ID is not enabled, the log files are written under the security context of the user ID of the person who is performing the action. In either case, the ID must have the proper authority to be able to write to this directory. Note: event logging uses the permission value that is specified by the ENABLE_ALTID_USS_SECURITY option in the Option Features Table “ENCOPTBL”. Where this option is not enabled, the default permission value for End-to-End logging files is “700”.</p><p>The next step is to set up the Mainframe Webhook Server component to watch for event files at the various locations you are logging them. As files are written, they will be read, posted as a webhook to any defined end-point URLs and finally the files will be deleted. The webhook server is a Java *.war or web archive file that is meant to be deployed to an application server like Tomcat. If you are running Endevor web services, you can deploy to the same Tomcat server the web services are deployed on. This is as simple as copying the mfwebhookserver.war file from the install media to the &lt;TOMCAT_HOME&gt;/webapps directory. One thing that should be noted though is that the event logs will be written to the USS directories in EBCDIC. As such, your JCL for running Tomcat should be adjusted so that the following Java option is included in the start up command:</p><pre>-Dmf.webhook.file.encoding=CP1140</pre><figure><img alt="Adjusted JCL for Webhook Server Deployment" src="https://cdn-images-1.medium.com/max/700/0*cX8hbwP8nF8YuouO" /><figcaption>Adjusted JCL for Webhook Server Deployment</figcaption></figure><p>Detailed instructions on deploying the webhook server can be found <a href="https://techdocs.broadcom.com/us/en/ca-mainframe-software/devops/ca-endevor-integrations-for-enterprise-devops/1-0/mainframe-webhook-server.html">here in the CA Endevor documentation under the “Mainframe Webhook Server” topic</a>.</p><p>Note: again, if you are not the Endevor administrator, you will likely need to enlist them to set this up. It may also already be set up in which case you simply need to ask them to provide the URL for the mainframe webhook server so you can configure the webhooks.</p><p>Once you’ve deployed the webhook server, you can go to the URL of your tomcat, with the mfwebhookserver context root (e.g. <a href="https://my.tomcat.host.name:port/mfwebhookserver)">https://<em>my.tomcat.host.name</em>:<em>port</em>/mfwebhookserver)</a> and create a new hook:</p><figure><img alt="Setting up an Endevor Webhook" src="https://cdn-images-1.medium.com/max/892/0*WW-SjYkJxDGv8QNv" /><figcaption>Setting up an Endevor Webhook</figcaption></figure><p>Make sure to specify the directory you are logging events to in Endevor in the “Watched Directory” field and the URL of the logstash listener in the “Payload URL” field. Check the activate box and click “OK”. Now events being logged by CA Endevor will be posted to the ELK stack and you can begin to add visualizations, alerts and get insights into your Endevor installation.</p><p><strong>Creating a Simple Visualization</strong></p><p>The first step to creating a visualization is to create an Index Pattern for your data so that it becomes visible to Kibana. This is done via the “Stack Management” menu item in the ELK web interface (<a href="http://my.server.fully.qualified.host.name:5601/">http://<em>my.server.fully.qualified.host.name</em>:5601/</a>) after you click the menu icon on the top left. If this is the first time, accessing the interface, you may see a prompt to set up some initial data. If you see this, click “Explore on my own”. Once you click the menu icon and select “Stack Management” from the list you will be in the Stack Management page. Click “Index Patterns” from the list of sections on the left (under the Kibana heading). Now click “Create Index Pattern”. For the name, enter <em>logstash*</em> and click “Next Step”. On the next page, select @timestamp in the “Time field” drop down and then click “Create Index Pattern”. At this point your data will be visible to Kibana.</p><p>Now if you go back to the Kibana home page (click the Elastic Icon on the top left and then click the Kibana tile to go to the homepage), you can create a dashboard by clicking on the “Dashboard” tile. Then click the “Create Dashboard” button and “Create Visualization” on the following screen. You should see your index pattern on the left, along with fields that were detected from the various webhook payloads. Let’s say you are working with package webhook data. A first visualization you could create is the breakdown of package activity over various timespans. This could be done by dragging the <em>@timestamp</em> field to the “Horizontal Axis” target on the right and then the <em># Records</em> field to the “Vertical Axis” target and finally the <em>action.keyword</em> to the “Break down by” target. Now you should have something like this:</p><figure><img alt="Creating a Simple Visualization" src="https://cdn-images-1.medium.com/max/1024/0*IIV3NboBW2iulIdP" /><figcaption>Creating a Simple Visualization</figcaption></figure><p>Of course, this is just a starting point. You can add other panels to the Dashboard and other visualizations. You can save the Dashboards for easy retrieval and create thresholds and alerts. From here, I recommend experimenting with the visualizations and seeing what use cases you can come up with.</p><p><strong>Summary</strong></p><p>CA Endevor, like any other system in your tool chain contains a large amount of valuable data that can help you get insights into your DevOps processes. Tools like the Elastic stack can be invaluable for analyzing this data and realizing important insights. With the possibilities afforded by Zowe, the CA Endevor REST API and webhooks, you can easily integrate Endevor into these tools as detailed above.</p><p>Learn more about <a href="https://www.ca.com/us/open-source-mainframe-zowe.html">Zowe at this site</a>; more <a href="https://medium.com/zowe">Zowe blogs here</a>.</p><p>If you have any questions or feedback or would like to talk about any of your <a href="https://medium.com/modern-mainframe/ca-endevor/home">CA Endevor use cases</a>, please feel free to contact me at <a href="/service/mailto:Vaughn.Marshall@broadcom.com">Vaughn.Marshall@broadcom.com</a> or leave a comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d76556864773" width="1" height="1" alt=""><hr><p><a href="https://medium.com/modern-mainframe/observability-for-ca-endevor-with-the-elastic-elk-stack-d76556864773">Observability for CA Endevor with the Elastic (ELK) Stack</a> was originally published in <a href="https://medium.com/modern-mainframe">Modern Mainframe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing Broadcom Team Build]]></title>
            <link>https://medium.com/modern-mainframe/introducing-ca-endevor-team-build-dab73251040f?source=rss-c3b69c22bfff------2</link>
            <guid isPermaLink="false">https://medium.com/p/dab73251040f</guid>
            <category><![CDATA[git]]></category>
            <category><![CDATA[gitlab]]></category>
            <category><![CDATA[cobol]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[mainframe]]></category>
            <dc:creator><![CDATA[Vaughn Marshall]]></dc:creator>
            <pubDate>Tue, 20 Apr 2021 15:13:11 GMT</pubDate>
            <atom:updated>2026-08-04T18:06:48.902Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introducing Team Build</h3><p>IDC concludes that organizations that modernize their mainframe software and processes have much better outcomes than those that choose to leave the platform altogether¹. This data includes adopting modern practices such as DevOps and Agile for maintaining mainframe application code. However, while the outcomes of modernization vs re-platforming are clear, determining the best way to modernize is not. Should an organization migrate to open source tools that their distributed counterparts use? Should they consider migrating all their code to a modern VCS like Git, or are there ways to modernize processes in-place and use new tools without such costly lift and shift migrations? To give our clients optimum flexibility when making these decisions, we’ve introduced a new tool: <em>Team Build</em>.</p><p>Endevor Team Build lets developers create a build project based on existing Endevor build automation, but which runs outside the traditional Endevor infrastructure. All that’s needed is a lightweight engine that can be deployed to any LPAR, including virtual LPARs, without the need for system programmer assistance. This is significant because there are situations where modernization-in-place doesn’t fit so flexibility to address all scenarios is key. Endevor Team Build, in conjunction with tools like <a href="https://mainframe.broadcom.com/devops">Bridge for Git and Zowe CLI</a>, gives Endevor that flexibility.</p><p>Let’s have a look at the different approaches and where each might fit.</p><figure><img alt="CA Endevor Configurations for Source Code, Build and Ship" src="https://cdn-images-1.medium.com/max/639/0*6m3U3l_uFJI3WW7p" /><figcaption>CA Endevor Configurations for Source Control, Build and Ship</figcaption></figure><p>Endevor Bridge for Git and Zowe CLI allow modern DevOps tools to be used as interfaces to underlying traditional Endevor repositories. This means the years of investment in automation such as complex build processes continues to deliver value uninterrupted. In fact, you can add new automation right on top of the old to take advantage of modern practices like static application analysis. There are other concrete advantages — because it’s only adding a new interface to existing technology, the risk and cost of changing tools is avoided. For monoliths that would be difficult to unwind or which have poorly defined application boundaries, even if there are hundreds of thousands of files — have only the working set of files made available to Git while the other build dependencies come from within the traditional repo. Most importantly, organizations don’t need to mandate any particular way of working. Are you a traditional developer who would much prefer to use ISPF and 3270 interfaces? No problem — work the way you always have. The tools allow collaboration with next generation developers and those who prefer to use tools like Git without forcing traditional developers to change the way they work.</p><p>So there are clear pros to modernizing in-place, but is that always going to be the right choice? The answer is not always cut and dried. What should developers do if they want to build on virtual LPARs? Tools like zD&amp;T let developers access their own virtualized mainframes running on x86 hosts to build and test their code. Use cases featuring browser-based IDEs paired with virtualized workspaces are becoming more common, including options for virtualized z/OS workspaces. Many of the enterprise Git providers are now hosting a browser based IDE that can be launched to work on your code branch. So while building on zD&amp;T may not be common right now, this is definitely something we will see more of in the near future. Does that mean that the full Endevor application, along with all its infrastructure, application code and dependencies, needs to be installed on the virtual LPAR to do builds? In this instance, it clearly doesn’t make sense.</p><p>What about other developer-centric use cases? Sometimes developers want to have more control over their builds, to change compiler options on demand as an example. While that can be accommodated in a traditional Endevor setup, it involves work and maintenance by the Endevor administrator. Bridge for Git also has a couple of limitations. In particular, there are lifecycle considerations that need to be observed to set it up for developer builds of their working directories. In some cases, organizations may not wish to make the necessary changes but with the introduction of Endevor Team Build, these use cases are covered as well.</p><p>Modernization decisions can be made on an application-by-application basis. One size does NOT need to fit all! Some applications are rarely if ever updated. Why invest any effort in modernization? Probably not much ROI there and you can continue to use traditional methods. Application monoliths or a team with a mix of developers who want to use traditional interfaces vs Git? If so, no problem — leverage Bridge for Git. Or, if there’s a need to build on zD&amp;T or to perform developer builds using Bridge for Git without lifecycle constraints, Endevor Team Build is there to help.</p><p>Hybrid use cases are also supported. If there are application dependencies maintained by teams using Endevor in traditional ways, Bridge for Git can be used to feed those dependencies to Team Build projects. Teams can also choose to use Team Build for Git-native collaboration until the code is ready to go to production. By merging on to designated branches, Bridge for Git can feed Endevor for the “production” build which is especially relevant if your change control processes are heavily tied to Endevor automation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/667/1*vVO_g5GYurI_ey1Ra_pb-w.png" /><figcaption>CA Endevor Configurations and Use Cases</figcaption></figure><p>In the end, the answer to the question “Which is the best way to modernize?” is that it is up to the teams to decide. There is no one clear direction to take — it depends on the makeup of the application team and their specific requirements. That’s why our tools need to support a wide array of options. I’ve always liked to compare Endevor to a toolbox — sometimes you need a screwdriver to build your solution and other times you’ll need a hammer (and more often than not you need both!). With the introduction of Endevor Team Build, you now have one more tool in the most comprehensive toolbox on the market.</p><p>To see Team Build in action, watch this <a href="https://www.youtube.com/watch?v=NxDR5DpUpNs">GitHub for Mainframe Demo</a>.</p><p>If you have any questions or feedback or would like to talk about any of your Endevor use cases, please feel free to contact me at <a href="/service/mailto:Vaughn.Marshall@broadcom.com">Vaughn.Marshall@broadcom.com</a>.</p><p>¹From <a href="https://idcdocserv.com/US46775720">“The Quantified Business Benefits of Modernizing IBM Z and IBM i to Spur Innovation”</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dab73251040f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/modern-mainframe/introducing-ca-endevor-team-build-dab73251040f">Introducing Broadcom Team Build</a> was originally published in <a href="https://medium.com/modern-mainframe">Modern Mainframe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using CA Endevor to Version zUnit Artifacts]]></title>
            <link>https://medium.com/modern-mainframe/using-ca-endevor-to-version-zunit-artifacts-e854f50873ec?source=rss-c3b69c22bfff------2</link>
            <guid isPermaLink="false">https://medium.com/p/e854f50873ec</guid>
            <category><![CDATA[unit-testing]]></category>
            <category><![CDATA[zowe]]></category>
            <category><![CDATA[mainframe]]></category>
            <category><![CDATA[cobol]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Vaughn Marshall]]></dc:creator>
            <pubDate>Tue, 16 Feb 2021 12:03:24 GMT</pubDate>
            <atom:updated>2022-02-01T20:20:29.370Z</atom:updated>
            <content:encoded><![CDATA[<h3>Mainframe Unit Testing with zUnit</h3><p>A best practice for automated testing is to store the tests themselves with the source code, in a version control system. This enables you to easily retrieve the tests that go with the version of code you are testing at any point in time to run the tests. This particularly true when you are talking about unit tests.</p><p>In the world of mainframe there are a number of different solutions for unit testing, one of them being IBM’s zUnit, which uses the xUnit architecture to allow the authoring of tests for COBOL and PL/I code. Following the xUnit architecture allows the tests called and their results to be captured in a variety of popular DevOps tools and IDEs.</p><h3>Versioning zUnit Artifacts in CA Endevor</h3><p>zUnit produces a number of different artifacts for each test. For example, it creates files with the test data, it creates the tests themselves and it also generates stubs for programs that are called. One challenge that customers often ask me about: the artifacts generated in a given test can have the same names as in other tests, but different contents — given that, how are these artifacts best versioned in CA Endevor? (Note: this issue is not present with the new <em>IBM Dynamic Test Runner for z/OS</em> introduced in IDz v14.2.1)</p><p>Artifacts in Endevor are classified under Systems, Subsystems and Types. It makes sense that the test artifacts would be in their own types but how do we deal with artifacts for different tests having the same name? You might be able to put each test in it’s own subsystem but that would be a lot of administrative overhead.</p><figure><img alt="Serializing Test Artifacts for CA Endevor Storage" src="https://cdn-images-1.medium.com/max/687/0*RSdlDCgIyH3lGe8Q.png" /><figcaption>Serializing Test Artifacts for CA Endevor Storage</figcaption></figure><p>One solution to this problem would be to serialize the artifacts for a given test into a single file before adding it to Endevor. By doing so, you can store each test in the same system/subsystem structure the code uses, but under a dedicated “test” type. When you need to run the tests, you would retrieve them from Endevor, and for each test, you would deserialize the artifacts to their original format and structure, run the tests and clean up afterwards. To ensure no name collisions are created, each test should be initially created (and later deserialized) under its own high-level qualifier so artifacts with the same name don’t overwrite each other.</p><h3>Serializing and Deserializing zUnit Tests for Management under CA Endevor</h3><p>So how would we go about serializing and deserializing the artifacts? We’ve come up with a utility that can do just that — a simple REXX script that can serialize a set of PDS’s under a given high-level qualifier to a single file or do the reverse to deserialize it.</p><p>It works as follows:</p><ul><li>Allows the selection of a dataset high-level qualifier, a single testcase and optional additional members using the COMPRESS JCL.</li><li>Reads the zUnit (AZUGEN) dataset to find files and members that are related to the single testcase.</li><li>Serializes related files and members, and any manually selected members, to an output file in an encrypted format. This output file can then be checked into Endevor.</li><li>Later, when the test needs to be run, the serialized file is extracted to a different high-level qualifier that is provided in the EXPAND JCL.</li></ul><p>You can find a <a href="https://github.com/BroadcomMFD/broadcom-product-scripts/tree/main/endevor/zunit">sample of the script &amp; JCL here</a>.</p><p>Note: while we’ve made every effort to ensure this works with zUnit artifacts and are confident it does, this utility is provided as is and should be considered field-developed and not supported as part of CA Endevor. You are welcome to use it as is and we hope it proves useful for working with zUnit and CA Endevor.</p><h3>Automating your Tests</h3><p>You may also be wondering how to best make this part of your automated pipeline? One option is to leverage Zowe CLI. Zowe CLI is a framework for executing mainframe operations via a command line interface. The CA Endevor plugin for Zowe CLI lets you perform Endevor operations from the command line. With a Zowe CLI script, you can extract the tests from CA Endevor, deserialize them and then submit them for execution, cleaning up once the tests are done. Here is a <a href="https://github.com/BroadcomMFD/broadcom-product-scripts/blob/main/endevor/zunit/Jenkinsfile">sample script that does this for Jenkins</a></p><h3>Summary</h3><p>A quick recap of what we’ve covered:</p><ul><li>zUnit is an IBM tool for creating COBOL and PL/I unit tests</li><li>zUnit has the potential to create name collisions between test artifacts</li><li>A utility to serialize / deserialize the artifacts will help you version them in CA Endevor and keep the tests with the code</li><li>The utility and the tests can be incorporated into your automation and CI/CD pipelines</li></ul><p>Hopefully this has helped solve some of the challenges associated with unit testing with zUnit code.</p><p>Learn more about <a href="https://www.ca.com/us/open-source-mainframe-zowe.html">Zowe at this site</a>; more <a href="https://medium.com/zowe">Zowe blogs here</a>.</p><p>For more information on the CA Endevor Plug-in for Zowe CLI visit: <a href="https://techdocs.broadcom.com/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-1/ca-endevor-scm-plug-in-for-zowe-cli.html">CA Endevor Zowe CLI Plug-in Documentation.</a></p><p>If you have any questions or feedback or would like to talk about any of your CA Endevor use cases, please feel free to contact me at <a href="/service/mailto:Vaughn.Marshall@broadcom.com">Vaughn.Marshall@broadcom.com</a>.</p><p><strong><em>Special thanks to Dan Walther for assistance with the scripts.</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e854f50873ec" width="1" height="1" alt=""><hr><p><a href="https://medium.com/modern-mainframe/using-ca-endevor-to-version-zunit-artifacts-e854f50873ec">Using CA Endevor to Version zUnit Artifacts</a> was originally published in <a href="https://medium.com/modern-mainframe">Modern Mainframe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Scaling CA Endevor Web Services with Multiple Instances]]></title>
            <link>https://medium.com/modern-mainframe/scaling-ca-endevor-web-services-with-multiple-instances-cf2cd9420cc3?source=rss-c3b69c22bfff------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf2cd9420cc3</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[web-services]]></category>
            <category><![CDATA[mainframe]]></category>
            <category><![CDATA[zowe]]></category>
            <category><![CDATA[load-balancing]]></category>
            <dc:creator><![CDATA[Vaughn Marshall]]></dc:creator>
            <pubDate>Fri, 21 Aug 2020 11:38:43 GMT</pubDate>
            <atom:updated>2020-08-25T11:43:25.432Z</atom:updated>
            <content:encoded><![CDATA[<h4>Sometimes a Single Web Service Instance is Not Enough…</h4><p>In the last couple of years, the number of options for interacting with CA Endevor off-host has grown dramatically. As more and more traffic comes from outside the mainframe environment, it is becoming critical to ensure that the APIs that service those applications are stable and able to scale to support larger and larger loads. Broadcom has made a number of improvements in the services themselves to address this: support for 64 bit Java and therefore larger heap memory, streaming of results and various other performance improvements. Despite this, sometimes the best way to deal with increased load is to scale horizontally by having multiple instances. Another benefit to this approach is you can increase up time by ensuring that when one instance goes down, others can take over and continue to provide service.</p><figure><img alt="Load Balanced CA Endevor Web Services with Zowe API Mediation Layer" src="https://cdn-images-1.medium.com/max/782/1*P7gy_LyEUifCEWF6ujW4CA.png" /></figure><h4>Added Instances = Added Maintenance?</h4><p>You may be afraid that all of those instances might have impacts on maintenance. For instance, would you then need to update each one of the installation locations whenever there is a patch or new version released? Won’t that increase down time? In this case, one of our customers came up with an interesting solution to this and they agreed to let us share their configuration with you. With their set up, only a single directory containing the web services software is required and it can be used to launch multiple instances. This means less disk is used and maintenance becomes much easier. The following describes their configuration and a few different options for achieving the goal of running parallel, load balanced instances of the Endevor web services from a single install.</p><h4>Customizing web services software files</h4><p>To start an Endevor web service task on each LPAR you will need multiple versions of the ENDEVOR.cfg file so that the spawned task gets started on the same LPAR as the web service task that spawned it. More on this later.</p><p>Most customization that you make to the web service files can be shared in any instance, e.g. tomcat-users.xml or context.xml.</p><p>For this solution you need to update /tomcat/conf/context.xml to allow symbolic linking by adding these lines:</p><pre>&lt;Resources allowLinking=”true”&gt;<br>&lt;/Resources&gt;</pre><p>As an Endevor administrator this customized version should of course be added to Endevor so rename the delivered version of context.xml and create a symbolic link:</p><pre>mv context.xml context.xml.vanilla</pre><pre>ln -fs &lt;endevor_dir&gt;/context.xml &lt;ws_software_dir&gt;/tomcat/conf/context.xml</pre><h4>Implementing single Endevor web service software stack for multiple sysplexes</h4><p>This solution makes use of the system variable &amp;SYSPLEX and allows you to copy the same software stack to multiple sysplexes without changing the contents of the directories when copying.</p><p><strong>server.xml</strong></p><p>server.xml contains the IP address and port of the Endevor web service so this is unique to each web service instance.</p><p>In order to avoid the need to edit server.xml when copying the web service software from one plex to the next, you can create a symbolic link with the system variable &amp;SYSPLEX, then create a uniquely named version of server.xml for each plex.</p><pre>mv server.xml server.xml.vanilla</pre><pre>ln -fs £SYSSYMR/&amp;SYSPLEX.server.xml.syml &lt;ws_software_dir&gt;/tomcat/conf/server.xml</pre><p>If you don’t use Endevor to save your customization, then create one server.xml file per plex:</p><p>&lt;plexname&gt;.server.xml.syml</p><p>If you save your customization in Endevor then add a secondary symbolic link:</p><pre>ln -fs &lt;endevor_dir&gt;/server.xml-&lt;plex-meaningful-name&gt; &lt;ws_software_dir&gt;/tomcat/conf/&lt;plexname&gt;.server.xml</pre><p>Create one server.xml file per plex and add them to Endevor.</p><p>For server.xml, &lt;ws_software_dir&gt;/tomcat/conf would now contain:</p><pre>server.xml                 Syml -&gt; £SYSSYMR/&amp;SYSPLEX.server.xml.syml<br>server.xml.vanilla         File<br>PLEX1.server.xml.syml      Syml -&gt; &lt;endevor_dir&gt;/server.xml-PLEX1<br>PLEX2.server.xml.syml      Syml -&gt; &lt;endevor_dir&gt;/server.xml-PLEX2</pre><p>On PLEX1 &lt;endevor_dir&gt;/server.xml-PLEX1 will be used</p><p>On PLEX2 &lt;endevor_dir&gt;/server.xml-PLEX2 will be used</p><p>You will also need individual LPAR or plex directories for the logs:</p><pre>&lt;ws_software_dir&gt;/conf/logs     Syml -&gt; &lt;ws_software_dir&gt;/conf/lpar<br>&lt;ws_software_dir&gt;/conf/lpar     Syml -&gt; £SYSSYMR/&amp;SYSNAME<br>LPAR1                           Dir<br>LPAR2                           Dir<br>LPAR3                           Dir</pre><p>Or the following if you want to use a link to an STC user directory for the logs:</p><pre>&lt;ws_software_dir&gt;/conf/logs     Syml -&gt; &lt;stc-home-dir&gt;/logs/lpar<br>&lt;stc-home-dir&gt;/logs/lpar        Syml -&gt; £SYSSYMR/&amp;SYSNAME<br>LPAR1                           Dir<br>LPAR2                           Dir<br>LPAR3                           Dir</pre><h4>Implementing Endevor web service on a multi LPAR sysplex with shared DASD</h4><p>This solution makes use of the system variable &amp;SYSNAME (LPAR name) and allows you to use the same software stack for multiple LPARs.</p><p><strong>ENDEVOR.cfg</strong></p><p>ENDEVOR.cfg contains the LPAR name so is unique to each LPAR, this is the “STC HostName” variable and defines the hostname/LPAR where the spawned task will be submitted.</p><pre>mv ENDEVOR.cfg ENDEVOR.cfg.vanilla</pre><pre>ln -fs £SYSSYMR/&amp;SYSNAME.ENDEVOR.cfg.syml &lt;ws_software_dir&gt;/webapps/endevor/ ENDEVOR.cfg</pre><p>If you don’t use Endevor to save your customization, then create one ENDEVOR.cfg file per LPAR:</p><p>&lt;LPAR_name&gt;.ENDEVOR.cfg.syml</p><p>If you save your customization in Endevor then add a secondary symbolic link:</p><pre>ln -fs &lt;endevor_dir&gt;/server.xml-&lt;plex-meaningful-name&gt; &lt;ws_software_dir&gt;/tomcat/conf/&lt;plexname&gt;.server.xml</pre><p>Create one ENDEVOR.cfg file per LPAR and add them to Endevor.</p><p>For ENDEVOR.cfg, &lt;ws_software_dir&gt;/webapps/endevor/ would now contain:</p><pre>ENDEVOR.cfg              Syml -&gt; £SYSSYMR/&amp;SYSNAME..ENDEVOR.cfg.syml<br>ENDEVOR.cfg.vanilla      File<br>LPAR1.ENDEVOR.cfg.syml   Syml -&gt; &lt;endevor_dir&gt;/ENDEVOR.cfg-LPAR1<br>LPAR2.ENDEVOR.cfg.syml   Syml -&gt; &lt;endevor_dir&gt;/ENDEVOR.cfg-LPAR2<br>LPAR3.ENDEVOR.cfg.syml   Syml -&gt; &lt;endevor_dir&gt;/ENDEVOR.cfg-LPAR3<br>LPAR4.ENDEVOR.cfg.syml   Syml -&gt; &lt;endevor_dir&gt;/ENDEVOR.cfg-LPAR4</pre><h4>Load Balancing Endevor Web Services on a Multi-LPAR Sysplex Using Zowe API Mediation Layer</h4><p>One easy option for setting up load balancing with multiple Endevor instances is to use the <a href="https://www.broadcom.com/products/mainframe/product-portfolio/open-source-mainframe-zowe">Zowe API Mediation Layer</a>. With the mediation layer in place, you simply need to register each of the instances behind the API ML so they share the same serviceId as <a href="https://docs.zowe.org/v1-8-x/extend/extend-apiml/api-mediation-onboard-an-existing-java-rest-api-service_plain-java-enabler.html#configuring-your-service">described here</a>.</p><p>Once configured, the API ML will automatically balance the load between instances that share the same serviceId.</p><h4>Load Balancing Endevor Web Services on a Multi-LPAR Sysplex Using a z/OS Communications Server TCP/IP distributed VIPA</h4><p>Another option for load balancing of Endevor web service requests is to start up the Endevor web service Tomcat task on each LPAR of a multi-LPAR Sysplex.</p><p>Set up a z/OS Communications Server TCP/IP distributed VIPA with the SERVERWLM parameter.</p><blockquote>“If the DISTMethod SERVERWLM parameter is specified on the respective VIPADISTRIBUTE statement, the distributing stack selects from the available servers for a DVIPA/port and forwards connections based on a WLM recommendation indicating how well each server is executing on its system.”</blockquote><p>In other words, the request will be sent to the best performing LPAR where the Endevor web service task is running, and if one LPAR is performing badly no requests will be sent there providing that there is a better performing LPAR with an Endevor web service task available.</p><p>More information on the IBM TCP/IP SERVERWLM parameter can be found on <a href="https://www.ibm.com/support/knowledgecenter/SSLTBW_2.4.0/com.ibm.zos.v2r4.halz002/vipa_sys_dist.htm">this web page</a>.</p><h4>Summary</h4><p>A quick recap of what we’ve covered:</p><ul><li>Using multiple instances of CA Endevor web services helps to scale to high user loads, as well as provide high availability</li><li>You can load balance using Zowe API Mediation Layer or Sysplex capabilities</li><li>You can use a single software stack for all of the instances through clever use of symbolic links, reducing your maintenance work</li></ul><p>Learn more about <a href="https://www.ca.com/us/open-source-mainframe-zowe.html">Zowe at this site</a>; more <a href="https://medium.com/zowe">Zowe blogs here</a>.</p><p>If you have any questions or feedback or would like to talk about any of your CA Endevor use cases, please feel free to contact me at <a href="/service/mailto:Vaughn.Marshall@broadcom.com">Vaughn.Marshall@broadcom.com</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf2cd9420cc3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/modern-mainframe/scaling-ca-endevor-web-services-with-multiple-instances-cf2cd9420cc3">Scaling CA Endevor Web Services with Multiple Instances</a> was originally published in <a href="https://medium.com/modern-mainframe">Modern Mainframe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automating CA Endevor Packages With Zowe CLI]]></title>
            <link>https://medium.com/modern-mainframe/automating-ca-endevor-packages-with-zowe-cli-b5ce593d5ff8?source=rss-c3b69c22bfff------2</link>
            <guid isPermaLink="false">https://medium.com/p/b5ce593d5ff8</guid>
            <category><![CDATA[jenkins]]></category>
            <category><![CDATA[cicd]]></category>
            <category><![CDATA[zowe]]></category>
            <category><![CDATA[mainframe]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Vaughn Marshall]]></dc:creator>
            <pubDate>Wed, 18 Mar 2020 11:31:55 GMT</pubDate>
            <atom:updated>2021-05-10T12:22:29.885Z</atom:updated>
            <content:encoded><![CDATA[<h3>CA Endevor is a DevOps Tool</h3><p>CA Endevor is a market leading mainframe DevOps tool and has been for many years. The very name Endevor actually stands for <strong>En</strong>vironment for <strong>Dev</strong>elopment and <strong>O</strong>pe<strong>r</strong>ations. A key DevOps practice is being able to easily and continuously build changes and shepherd them through a defined life cycle or pipeline. Given that, it should be no surprise that CA Endevor provides this capability.</p><p>The compile process for mainframe applications does not generally produce bundles of executable code for deployment as a complete application. For example, with java-based applications, artifacts such as jars, wars and ears (i.e. complete application bundles) are created during the build and deployed to different environments at various stages in the pipeline. By contrast, the mainframe compile and link process produces load modules with loose dependencies. Tools like CA Endevor are used to identify the dependencies and bundle them, along with the executables, and move them through the pipeline and its associated environments. The vehicle for this in Endevor is called a <em>package</em>.</p><figure><img alt="CA Endevor incorporated into a CI/CD pipeline via package automation" src="https://cdn-images-1.medium.com/max/1024/1*H5M4qVit4fjUdx62aUuhAg.png" /></figure><h3>Packages Defined</h3><p>Traditionally, mainframe developers have had the responsibility of defining packages based on the changes they make. In many cases, the packages they use to promote from development and deploy to QA are not the same as the ones used for higher environments. Packages for higher environments need to account for dependencies as well as defect fixes moved in after the initial change. In the world of modern mainframe though, organizations are looking to automate this activity in the interest of improving velocity and relieving developers from this burden.</p><p>To understand how we can begin to automate packages for CA Endevor, let’s first look at what a package is. A package, simply put, is a series of instructions to do <em>something</em> in Endevor. The instructions are specified in a language called SCL. It can include things like adding or building elements but more typically though, they are a series of move instructions to advance changes through a defined life cycle. Developers generally decide what they want to move, then interactively build or create the package commands using a menu-driven interface that allows them to choose what to move, and from where. When they are ready to act on the instructions, they perform an operation called <em>cast</em> which validates the SCL and ensures required dependencies are included in the package and any potential version clashes have been resolved. After the cast operation, the package and the affected elements are frozen until the package instructions have completed. Once completed, the package is marked with the <em>Executed</em> status. The package may also require approvals before operations can begin.</p><p>In general, packages operate entirely within the CA Endevor managed datasets. Very frequently though, the binaries need to be delivered to additional, highly secured non-Endevor managed datasets after being moved up the life cycle in the Endevor environment. This can be achieved by <em>shipping</em> the package to an external destination known to and authorized for access by Endevor.</p><h3>Automating Packages in a Pipeline</h3><p>In terms of automation then, we need to be able to support the following operations:</p><ul><li>Defining the initial package</li><li>Updating package SCL — similar to what a developer might do interactively, where the SCL is newly created or added to existing SCL defined for the package</li><li>Casting the package</li><li>Approving the package — potentially by the pipeline itself after approval is obtained in another tool like a change control system</li><li>Executing the package</li><li>Shipping the package</li></ul><p>In a pipeline context you may very well want to repeat the above steps at each phase of the life cycle, woven in with non-Endevor tasks like change control operations or automated / manual testing. One of the nice gains you get from automating your package process is the fact you can put all these steps together as part of an end-to-end workflow, regardless of what platform the steps are performed on.</p><p><a href="https://www.broadcom.com/products/mainframe/devops-app-development/open-mainframe/open-source-mainframe-zowe">Zowe CLI </a>facilitates all of this by providing command line operations that can perform each of the required tasks. Simple shell scripting combined with an overall orchestration tool like Jenkins Pipeline creates the end-to-end workflow. If you are looking for a solution that requires less scripting, you can find purpose-built integrations similar to what is being described here for Broadcom’s own <a href="https://www.broadcom.com/products/software/automation/continuous-delivery/automic-continuous-delivery-director">Automic Continuous Delivery Director</a>, which is powered by Zowe CLI.</p><h3>Defining Packages with Zowe CLI</h3><p>Let’s look at each of the operations in detail, starting with defining a package and generating SCL for it. To define a package in Zowe CLI, you would use the following command:</p><p>zowe endevor create package <em>Package_Name</em> -d <em>Package_Description</em> -i <em>Endevor_Instance</em> --novalidate-scl</p><p>For example:</p><p>zowe endevor create package MYPKG -d &quot;Its a package&quot; -i ENDEVOR --novalidate-scl</p><p>Note: the --novalidate-scl — this is used to create an empty package initially so we can query for elements to add to the package and generate the SCL we will use.</p><p>You may be wondering how that command is connecting to the mainframe — for instance, what LPAR is this being run on and what user ID is used? Those are all things stored in a secure Zowe CLI <em>profile</em>. The profile tells Zowe CLI which Endevor web service host and port to connect to, along with some other parameters like username and password. The web service host, port and instance (the -i parameter above) effectively determine the LPAR you are running on. You can have multiple profiles and specify which one to use via the settings for default profile or you can pass it as a command line parameter. You can also specify the profile settings explicitly on the command line if preferred. To review the capability around profiles, use the Zowe CLI profiles command group:</p><p>zowe profiles --help</p><p>In order to create the SCL to use with the package, you first need to get some details about the elements you want to add. Developers will typically know the elements they changed and be able to find them quickly to build SCL in ISPF, but in an automated setting, the human element is not present. This means we need some sort of way of finding out what we changed. A natural candidate for that is the Endevor <em>CCID </em>(<strong>C</strong>hange <strong>C</strong>ontrol <strong>ID</strong>entifier)<em> </em>of the change. A CCID is a tag that is meant to associate an element update with the ID of a related change control artifact in another system. We can query for changes in a given inventory location in Endevor and use the results to build the SCL. If our pipeline allows parameterization, we might parameterize the CCID or even other inventory location values or maybe we will hard code the inventory locations for our application in our pipeline.</p><p>To search for elements by CCID, you would use the zowe endevor list ele command. In addition to the inventory location parameters (e.g. --environment --system --subsystem --stage-number --type), you can add filters for CCID as follows:</p><figure><img alt="Zowe CLI list elements command help showing CCID filter options" src="https://cdn-images-1.medium.com/max/814/1*nGyYnVzVQNA-wJLyp0EbHQ.png" /></figure><p>So for instance, if you wanted to find all the cobol elements that were changed with CCID = US227 for the last action, in system = FINANCE, with subsystem = ACCT* in environment = SMPLTEST and stage number = 1, you could use the following zowe command:</p><p>zowe endevor list elements --env SMPLTEST --sn 1 --sys FINANCE --sub ACCT* --type COBOL --where-ccid-lastact US227 --sm -i ENDEVOR</p><p>The output from the command will look something like this:</p><pre>FINARP01 COBOL SMPLTEST 1 FINANCE ACCTREC<br>FINARP02 COBOL SMPLTEST 1 FINANCE ACCTREC<br>FINARP03 COBOL SMPLTEST 1 FINANCE ACCTREC<br>FINARS03 COBOL SMPLTEST 1 FINANCE ACCTREC<br>FINARS04 COBOL SMPLTEST 1 FINANCE ACCTREC<br>….</pre><p>Of course, this is not SCL — so now we need to parse the output and turn it into proper SCL syntax before we update the package. HINT: The --sm in the example will make the output easier to parse to turn it into SCL.</p><p>A very common use case is to create packages to perform the “Move” action in Endevor. The syntax for moving an element is as follows:</p><pre>MOVE ELEMENT ‘<em>Element_Name</em>‘<br>FROM ENV ‘<em>Environment_Name</em>’ SYS ‘System_Name’ SUBS ‘<em>Subsystem_Name</em>’<br>TYP ‘<em>Type_Name</em>’ STAGE NUMBER <em>Stage_Number<br></em>.</pre><p>NOTE: there are other options such as CCID, Bypass Element Delete, With History or Retain Signout that you may wish to include. As well, the period shown above must be included at the end of each SCL statement. Details can be found <a href="https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-1/scl-reference/process-element-actions-in-batch/move-statement.html">here</a>.</p><p>HINT: The syntax for building elements is very similar to the above, essentially substituting GENERATE ELEMENT for MOVE ELEMENT.</p><p>To turn the output from the list command to SCL, you would use a simple script, leveraging regex to parse the output and write a temporary SCL file with the correct SCL syntax. Here’s a sample of what that might look like in bash, starting from the define package command:</p><pre>#!/bin/bash<br>#<br># Define package script for a static app inventory location<br># and using a given package name and CC ID to query for elements<br>#<br># define_pkg.sh package_name cc_id<br>#</pre><pre># Define an empty package<br>zowe endevor create package $1 -d &quot;$1&quot; --novalidate-scl</pre><pre># Query for cobol elements by CC ID in FINANCE/ACCTREC in SMPLTEST/1<br># Store results in a temp file<br>TMPFILE=$(mktemp)<br>zowe endevor list elements --env SMPLTEST --sn 1 --sys FINANCE --sub ACCTREC --type COBOL --where-ccid-lastact $2 --sm &gt; $TMPFILE</pre><pre># Read the file and translate to an SCL syntax temp file<br>SCLFILE=$(mktemp)<br>regex=&#39;([[:alnum:]]*)[[:blank:]]*([[:alnum:]]*)[[:blank:]]*([[:alnum:]]*)[[:blank:]]*([[:digit:]])[[:blank:]]*([[:alnum:]]*)[[:blank:]]*([[:alnum:]]*)&#39;<br>while IFS= read -r line; do<br>    [[ &quot;$line&quot; =~ $regex ]]<br>    echo &quot;MOVE ELEMENT ${BASH_REMATCH[1]}&quot; &gt;&gt; $SCLFILE<br>    echo &quot;  FROM ENV &#39;${BASH_REMATCH[3]}&#39; SYS &#39;${BASH_REMATCH[5]}&#39; SUBS &#39;${BASH_REMATCH[6]}&#39;&quot; &gt;&gt; $SCLFILE<br>    echo &quot;  TYPE ${BASH_REMATCH[2]} STAGE NUMBER ${BASH_REMATCH[4]}&quot; &gt;&gt; $SCLFILE<br>    echo &quot;.&quot; &gt;&gt; &quot;$SCLFILE&quot;<br>done &lt; &quot;$TMPFILE&quot;</pre><pre>zowe endevor update package $1 --from-file &quot;$SCLFILE&quot;</pre><pre>rm &quot;$TMPFILE&quot;<br>rm &quot;$SCLFILE&quot;</pre><p>HINT: The above script is a simple shell script, however, you may use any scripting language you want with Zowe CLI, such as python or javascript. Most commands also allow you to pass the --rfj option that formats the output in JSON making for easy parsing in modern languages.</p><h3>Performing Package Operations with Zowe CLI</h3><p>Most of the other operations are simpler, with simple Zowe CLI commands mapping to each operation. For example to cast a package, you use the following command:</p><p>zowe endevor cast package <em>Package_Name </em>-i <em>Endevor_Instance</em></p><p>To approve a package, you would use this command:</p><p>zowe endevor approve package <em>Package_Name </em>-n “<em>Approval_Notes</em>” -i <em>Endevor_Instance</em></p><p>If you were rejecting the package, you change the approve action to “deny”:</p><p>zowe endevor deny package <em>Package_Name </em>-n “<em>Approval_Notes</em>” -i <em>Endevor_Instance</em></p><p>To execute the actions in a package, you would use the following command:</p><p>zowe endevor execute package <em>Package_Name </em>-i <em>Endevor_Instance</em></p><p>Note: for long running packages you can also use the zowe endevor submit package command. As well, for all the commands above, there are different options for each command I would encourage you to explore, depending on the needs for your pipeline.</p><h3>Shipping Packages with Zowe CLI</h3><p>The last operation to cover is Package Ship. The Package Ship facility is available in Zowe CLI via the zowe endevor submit scl command, which takes input from an SCL file on the machine the script is running on. This means we may need a bit more scripting to perform a ship operation.</p><p>Package Ship SCL has the following format to ship the original package binaries:</p><p>SHIP PACKAGE <em>Package_Name </em>TO DESTINATION <em>Destination_Name </em>OPTION OUTPUT .</p><p>Or, to ship the backout binaries, you would use:</p><p>SHIP PACKAGE <em>Package_Name </em>TO DESTINATION <em>Destination_Name </em>OPTION BACKOUT .</p><p>Note: multiple ship statements in a single submission is not allowed with the Ship API so you need to submit each one at a time. Let’s assume an application has Package Ship locations in Chicago, Dallas and New York. Here is a snippet of script that would ship a given package name (passed as a parameter to the script) to those locations via Zowe CLI:</p><pre>#!/bin/bash<br>#<br># Define package ship script for a static ship destinations<br># and using a given package name<br>#<br># ship_pkg.sh package_name <br>#<br># Create a temporary SCL file<br>SCLFILE=$(mktemp)</pre><pre># Run Ship SCL statements<br>echo &quot;SHIP PACKAGE $1 TO DESTINATION CHICAGO OPTION OUTPUT . &quot; &gt; $SCLFILE<br>zowe endevor submit scl --scl-file &quot;$SCLFILE&quot; --scl-type ship -i ENDEVOR</pre><pre>echo &quot;SHIP PACKAGE $1 TO DESTINATION DALLAS OPTION OUTPUT . &quot; &gt; $SCLFILE<br>zowe endevor submit scl --scl-file &quot;$SCLFILE&quot; --scl-type ship -i ENDEVOR</pre><pre>echo &quot;SHIP PACKAGE $1 TO DESTINATION NEWYORK OPTION OUTPUT . &quot; &gt; $SCLFILE<br>zowe endevor submit scl --scl-file &quot;$SCLFILE&quot; --scl-type ship -i ENDEVOR</pre><pre># Clean up<br>rm &quot;$SCLFILE&quot;</pre><p>HINT: The zowe endevor submit scl command will also allow other types of SCL to be run, including admin operations, making it a powerful way to incorporate CA Endevor operations that we don’t yet have dedicated commands for.</p><h3>Summary</h3><p>Hopefully this has given you a good idea of how you can start building package automation into your CI/CD pipelines. By doing so, you can:</p><ul><li>Create cross-platform pipelines while at the same time leveraging existing automation you already use for deployments</li><li>Weave in other tasks so the automation you have can become an end-to-end workflow, even when the added tasks you want to automate run on other platforms such as automated GUI testing</li><li>Use the visualization and auditing that comes with modern CI/CD tools</li><li>Use webhook consumption to respond to events and drive downstream processing</li></ul><p>Learn more about <a href="https://www.ca.com/us/open-source-mainframe-zowe.html">Zowe at this site</a>; more <a href="https://medium.com/zowe">Zowe blogs here</a>.</p><p>For more information on the CA Endevor Plug-in for Zowe CLI visit: <a href="https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-0/using-the-ca-brightside-plug-in-for-ca-endevor-scm.html">CA Endevor Zowe CLI Plug-in Documentation.</a></p><p>As well, if you have any comments, questions or suggestions on the above, I’d love to hear from you — send me an email at <a href="/service/mailto:Vaughn.Marshall@broadcom.com">Vaughn.Marshall@broadcom.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b5ce593d5ff8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/modern-mainframe/automating-ca-endevor-packages-with-zowe-cli-b5ce593d5ff8">Automating CA Endevor Packages With Zowe CLI</a> was originally published in <a href="https://medium.com/modern-mainframe">Modern Mainframe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting up CA Endevor for Zowe CLI]]></title>
            <link>https://medium.com/zowe/setting-up-ca-endevor-for-zowe-cli-d8833a8312ee?source=rss-c3b69c22bfff------2</link>
            <guid isPermaLink="false">https://medium.com/p/d8833a8312ee</guid>
            <category><![CDATA[zoweecosystem]]></category>
            <category><![CDATA[git]]></category>
            <category><![CDATA[mainframe]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[zowe-cli]]></category>
            <dc:creator><![CDATA[Vaughn Marshall]]></dc:creator>
            <pubDate>Wed, 19 Feb 2020 21:26:04 GMT</pubDate>
            <atom:updated>2021-11-16T16:27:04.939Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>{</strong><a href="https://github.com/zowe/community/blob/master/blogging/blog_guidelines.md#zowe-ecosystem"><strong>Ecosystem</strong></a><strong>}</strong> Zowe is modernizing the way z/OS services are integrated with off-platform services. Since Zowe is open source and extensible, 3rd party tools can use Zowe as a modern access and integration path. CA Endevor is one example of that. As the leading SCM on z/OS, integration of CA Endevor processes with CI/CD tools is of utmost importance to many users in the world of DevOps. If you’d like to see how other z/OS products are also using Zowe to modernize their products, here’s a <a href="https://www.openmainframeproject.org/projects/zowe/conformance">full list</a> of Zowe conformant extensions.</p><p>At the core of Zowe are infrastructure APIs which can be used to build CLIs, GUIs or any kind of custom applications. Once you build APIs for a z/OS service, the possibilities are endless for modernization.</p><p>CA Endevor was actually ahead of the game and had already built APIs even before the advent of Zowe. However, with Zowe came the <a href="https://medium.com/zowe/zowe-and-mainframe-apis-74288cd0489b">API mediation layer</a> that manages infrastructure APIs on z/OS. Since Endevor is part of the core infrastructure, the decision was made to integrate the Endevor APIs with the Zowe API mediation layer. Endevor’s APIs had to be modified a little bit according to Zowe’s API conformance rules to integrate with the Zowe API mediation layer but the effort was relatively small.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E9Q3MowwS8sw4vLAMdr7eA.png" /><figcaption>Connecting CA Endevor to Modern DevOps Tools via Zowe CLI</figcaption></figure><p>After the APIs were built, the Endevor team decided to build a <a href="https://medium.com/zowe/getting-started-with-zowe-cli-7a29420c6be7">Zowe CLI</a> plugin for Endevor. The <a href="https://www.npmjs.com/package/@broadcom/endevor-for-zowe-cli">Zowe CLI Endevor plugin</a> was the perfect tool to naturally integrate Endevor with CI/CD tools like Jenkins. It gives developers the freedom to build quick automation for common tasks like builds and deployments and integrate it into DevOps standards being used in their organizations.</p><p>The Zowe CLI plugin for Endevor kick-started an influx of modernization for Endevor as the APIs were now ripe for the picking. Here’s a list of other Endevor tools built on the Zowe conformant Endevor APIs:</p><ol><li>The “<a href="https://docs.broadcom.com/docs/endevor-software-change-manager-bridging-enterprise-git?_ga=2.149291510.1102376631.1580504027-1626903383.1580504027">CA Endevor Bridge for Git</a>” which allows the build and source control engineer to mirror source code from Endevor in GitHub and BitBucket (and in the very near future, GitLab) so developers can access source through git.</li><li>The <a href="https://marketplace.visualstudio.com/items?itemName=broadcomMFD.explorer-for-endevor">“Explorer for Endevor” extension</a> which allows access to the full map of elements in Endevor in Microsoft Visual Studio Code or Eclipse Che</li></ol><p>So, are you ready to modernize your Endevor experience?</p><p>Get started by setting up the CA Endevor APIs and connecting them to the Zowe API mediation layer (you may need your mainframe IT staff’s help with the following):</p><ol><li>Ensure you’re on Endevor V18.0.12 or higher and CA Common Services 14.0 or higher, with the appropriate maintenance.</li><li>Common Services <a href="https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-0/installing/how-to-enable-web-services.html#concept.dita_e51fea82e410f5d772daa482909b79c07d125517_Heading">maintenance pre-reqs</a></li><li>Web Services <a href="https://techdocs.broadcom.com/us/en/ca-mainframe-software/devops/ca-brightside/3-0/zowe-cli/install-zowe-cli/install-cli-plug-ins/software-requirements-for-cli-plug-ins.html">maintenance pre-reqs for Zowe CLI</a></li><li><a href="https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-0/installing/how-to-enable-web-services/configure-ca-endevor-scm-for-web-services.html">Configure Endevor for Web Services</a></li><li><a href="https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-0/installing/how-to-enable-web-services/enable-stc-definitions-for-web-services.html">Configure CCI and ENF started task definitions</a></li><li><a href="https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-0/installing/how-to-enable-web-services/enable-security-access-to-web-services.html">Enable Security Access to Web Services</a></li><li><a href="https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-0/installing/how-to-enable-web-services/install-and-set-up-ca-endevor-scm-web-services.html">Install and Start the Endevor Web Services</a></li><li><a href="https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-mainframe-software/devops/ca-endevor-software-change-manager/18-1/installing/integrate-rest-api-with-the-api-mediation-layer.html">Integrate the REST API with the API Mediation Layer</a> (Optional Step)</li></ol><p><a href="https://attendee.gotowebinar.com/recording/5281282407568077825">We did a recent webinar</a> showing the process which may be of interest. There is also <a href="https://community.broadcom.com/mainframesoftware/communities/community-home/digestviewer/viewthread?MessageKey=486fb770-3b74-40d8-801c-3b99fe512232&amp;CommunityKey=592eb6c9-73f7-460f-9aa9-e5194cdafcd2&amp;tab=digestviewer#bm486fb770-3b74-40d8-801c-3b99fe512232">free web-based training on this topic</a> available from Broadcom for customers active on maintenance.</p><p>Next, install Zowe CLI and the Endevor plugin to get started with your automation and CI/CD efforts:</p><ol><li>Download Zowe and <a href="https://docs.zowe.org/stable/user-guide/cli-installcli.html">Install NodeJS and Zowe CLI</a></li><li>Add the <a href="https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-mainframe-software/devops/ca-brightside/3-0/ca-brightside-command-line-interface-cli/installing-cli/install-cli-plug-ins.html">CA Endevor plugin</a> into the Zowe CLI framework</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1005/0*9-_vZ1r-eUI6h43g" /><figcaption>Web Services Installation and Configuration Overview</figcaption></figure><p>At this point — you have everything you need to start using Zowe CLI with CA Endevor. Remember, even visualizing your Endevor actions like generates and package creations in a pipeline is of great value in building trust, standardization and quality in your dev teams.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d8833a8312ee" width="1" height="1" alt=""><hr><p><a href="https://medium.com/zowe/setting-up-ca-endevor-for-zowe-cli-d8833a8312ee">Setting up CA Endevor for Zowe CLI</a> was originally published in <a href="https://medium.com/zowe">Zowe</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>