CommunityDevOps ExchangePartners
Articles
12/19/2022
10 minutes

The CI/CD Pipeline: Why Testing Is Required at Every Stage

Written by
Team Copado
Table of contents

Originally published by New Context.

In the modern tech landscape, it is crucial to be able to quickly build and deploy code that you know will work. This is achieved through having a continuous integration and continuous delivery (CI/CD) pipeline that properly tests your application before deployment and monitoring it after deployment. In this series of articles on CI/CD, we are going to talk about how to test throughout the CI/CD pipeline, some CI/CD best practices, and we’ll conclude by waxing philosophical about CI/CD and providing some words of encouragement.

Does your code build properly? Does it do what you expect it to do? Testing is how you can programmatically determine the answer to these questions and more. With those answers, you can better automate your processes. If your tests are asking the right questions and you’re getting the right answers, do you need to look at every single result and give your approval so the code can deploy?

In order to truly know that your code is working, you must implement various kinds of testing throughout your deployment pipeline. When testing is a key component of your continuous integration and continuous delivery (CI/CD) implementation, you will be able to deliver with speed and reliability. When configured, your CI/CD pipeline will automatically build, test, and deploy your code when it is merged into a branch. However, even though your pipeline will have the major stages of build, test, and deploy, there is testing that happens throughout the pipeline. In this article, we will discuss what testing is available to you, what pipelines look like, and then discuss how to test throughout the pipeline.

What Kinds of Testing are Available Throughout the CI/CD Pipeline?

The types of tests you use, regardless of workload type (e.g., web app, container, backend application), are going to be largely the same. There will be special considerations for each language and platform and environment. Languages and platforms will use different tooling. Your environments will have different configurations, networks, and secrets. But the fundamental ideas will be the same.

Regardless of language or platform, you will need to test and those principles remain the same. Let’s run through some core types of testing:

  • Unit Testing: A unit test, as the name implies, tests an individual unit within your code: a function, module, or set of processes. By testing this way, you’ll be able to prove that each portion of the code is functioning correctly.
     
  • Integration Testing: Once you’ve proven that each unit of your code is functioning properly, you can then do integration testing to make sure that each unit is working properly with each other.
     
  • Performance Testing: This kind of testing is used to determine if your systems will be responsive and stable.

There are a number of tests you can perform to determine if your system is performant. Let’s cover a few of them:

  • Load Testing: Can your system handle the expected load? What about higher loads?
     
  • Stress Testing: Now that your system can handle high loads, let’s push the limits and see what it can really handle.
     
  • Spike Testing: This type of testing determines if the system can handle large and quick spikes in load.
     
  • Configuration Testing: What happens to the system if we start messing around with its configuration? How does this affect the performance?
     
  • Security Testing: Security testing checks for flaws in your project’s security by checking for risks and vulnerabilities.
     
  • Chaos Testing: This is an extreme form of testing that is used in production environments. Chaos testing will kill nodes, disable ports, and make other heinous configuration changes. The purpose of this chaos is to test the resiliency of your system and, to see if things can self-heal properly. A; and all of this testing is done during business hours so your developers can fix things while they’re at work, not at three in the morning.
Testing are Available Throughout the CI/CD Pipeline - Copado

Testing Throughout the CI/CD Pipeline

You might be asking yourself: What should I be testing for? How and when should I be doing it? Do I need to test every single line of code? We’ve run through some core testing methodologies, so now let’s run through a basic CI/CD pipeline and talk about when we should be applying those tests. First, let’s summarize the basic stages of a CI/CD pipeline and then get more specific:

  • Build: Depending on your specific application and environment configuration, your code is compiled, dependencies are pulled in and the application is built into a deployable artifact. Docker images may be built at this time.
     
  • Test: Automated tests are run against your code. These tests are written to ensure not only that your code is written correctly, but that it’s doing what you think it should be doing.
     
  • Deploy: Your code is deployed to your various environments. This can be fully automated. More often, the code is automatically deployed to the lower environments for testing. Then, in production, there may be needed approvals, a scheduled deployment, or, if all tests pass, an automated deployment.

Build

Your CI/CD pipeline always begins with a version control system. When your code is pushed to the remote repository, it should be automatically built. This could be building a Docker image, compiling code, running through an interpreted language. Even though this stage is going to look different for various types of applications, the core principle remains the same: the successful compilation or interpretation of your code is a test unto itself.

Test

Beyond that, you need to validate that your code is doing what you think it should be doing. This is where unit testing and integration testing come into play. Spend some serious time here and make sure that the results you’re getting are what you expect. Do you need to test for every single line of code? Absolutely not. In fact, having 100% code coverage is a fundamentally flawed idea because it works under the assertion that all of your tests are good. We are interested in testing for the right situations which will probably not be every line of code. Additionally, if your tests are too simple or too general, your functions can change in significant ways and the tests will still pass (remember, 1/1=1, but so does 1*1). Every test needs to be meaningful and serve a purpose.

Although your code is not yet deployed anywhere, there is some performance testing you can be doing now. Check for optimization and efficiency. Perform security testing to make sure that the base code isn’t vulnerable. Run through some configuration testing to make sure that your system is robust.

Do your tests need to vary based on the environment? Probably not, unless there are special or obvious caveats. But this is the exception, not the rule. Any differences between environments should be extracted into an environment configuration set and tested for.

Deploy

Once your code is merged into your main branch, it is ready for deployment to your various environments so that it can be tested, vetted, and ultimately proven worthy of production deployment. In this section we will discuss when to tag, merge, and release code. We will also briefly cover different deployment techniques.

When should you tag and release your code? Like all IT questions, it depends. If you’re using trunk-based development, all of your code will be merged into the main branch. When you are ready for a release, tag the main branch and run that tag through your testing environments before deploying to production. But how often should these releases happen? Again, that depends. For internal services, you can usually release more often. For critical and public systems, a more judicious approach may be warranted, perhaps only releasing on certain days or after certain times. But if your deployment methodology allows it, you can easily roll back to a known safe version which will allow you to deploy more frequently and with more confidence.

There are a variety of deployment methodologies that you will need to consider. Let’s cover some of the more common ones include:

Recreate strategy: Tear down the old and stand up the new. This is an antiquated approach and is listed here for historical reasons and as a cautionary tale. Think about every time you’ve gone to a website that is “Under construction.” Do you ever return? This methodology is dated and is only useful for losing customers.

Canary deployments: A canary deployment slowly rolls out your deployment to a small subset of your systems. This way, if there are issues, only a small percentage of your users and systems will be impacted. This methodology has the added benefit of zero downtime.

Blue/green deployments: With this methodology, you have two production environments which should be as identical as possible. One is live and receives the production traffic. For the purpose of this example, let’s say that blue is live. When you deploy your new release, you deploy to the other environment (in this case, green). Once deployed, you make your router point all of the traffic to the new environment. A big advantage of this approach is that if there are issues, you can easily roll back to the previous environment. And there is zero downtime.

A/B testing: This is more a form of testing than deployment, but it’s important enough to discuss here, especially having just discussed blue/green deployment. A/B testing is when your current production code is running on Production Environment A and you then deploy your new code to Production Environment B. Unlike blue/green testing, though, both A and B are live and receive traffic. This is a useful testing methodology because with both systems live, you can track users and usage and gain valuable insight about your changes.

Feature toggles: Also known as feature flags, this form of development provides a mechanism for developers to toggle features so that they can run, not run, or even be hidden during runtime. This methodology allows developers to easily rollout features to specific subsets of users, allowing for targeted roll outs.

The bulk of your performance testing will happen once your code has been deployed. Make sure that your application can handle the loads you expect. Ensure that it can handle spikes in traffic. If your application auto-scales, is that working? Does it scale back down when the traffic dies down? Really flog the application and see where it breaks. It’s always going to break at some point, you just need to make sure it breaks at a point you’re comfortable with.

Some testing, like chaos testing, happens in production; but not everyone is going to find the same levels of success from testing this way. It is worthwhile to spend some time, discuss these different strategies, and implement them. Even if you end up scrapping some forms of testing, you will have learned and documented what works and what doesn’t work in your environment.

Ensure that Your CI/CD Pipeline Testing Is Dynamic

Testing throughout the pipeline allows you not only to properly test your code, but can also help you speed up your deployment process. Not all tests have to be run serialized. Testing throughout the pipeline will help you parallelize.

 

Book a demo

About The Author

#1 DevOps Platform for Salesforce

We build unstoppable teams by equipping DevOps professionals with the platform, tools and training they need to make release days obsolete. Work smarter, not longer.

Copado Launches Copado Explorer to Simplify and Streamline Testing on Salesforce
Exploring Top Cloud Automation Testing Tools
Master Salesforce DevOps with Copado Robotic Testing
Exploratory Testing vs. Automated Testing: Finding the Right Balance
A Guide to Salesforce Source Control
A Guide to DevOps Branching Strategies
Family Time vs. Mobile App Release Days: Can Test Automation Help Us Have Both?
How to Resolve Salesforce Merge Conflicts: A Guide
Copado Expands Beta Access to CopadoGPT for All Customers, Revolutionizing SaaS DevOps with AI
Is Mobile Test Automation Unnecessarily Hard? A Guide to Simplify Mobile Test Automation
From Silos to Streamlined Development: Tarun’s Tale of DevOps Success
Simplified Scaling: 10 Ways to Grow Your Salesforce Development Practice
What is Salesforce Incident Management?
What Is Automated Salesforce Testing? Choosing the Right Automation Tool for Salesforce
Copado Appoints Seasoned Sales Executive Bob Grewal to Chief Revenue Officer
Business Benefits of DevOps: A Guide
Copado Brings Generative AI to Its DevOps Platform to Improve Software Development for Enterprise SaaS
Celebrating 10 Years of Copado: A Decade of DevOps Evolution and Growth
Copado Celebrates 10 Years of DevOps for Enterprise SaaS Solutions
5 Reasons Why Copado = Less Divorces for Developers
What is DevOps? Build a Successful DevOps Ecosystem with Copado’s Best Practices
Scaling App Development While Meeting Security Standards
5 Data Deploy Features You Don’t Want to Miss
Top 5 Reasons I Choose Copado for Salesforce Development
How to Elevate Customer Experiences with Automated Testing
Getting Started With Value Stream Maps
Copado and nCino Partner to Provide Proven DevOps Tools for Financial Institutions
Unlocking Success with Copado: Mission-Critical Tools for Developers
How Automated Testing Enables DevOps Efficiency
How to Keep Salesforce Sandboxes in Sync
How to Switch from Manual to Automated Testing with Robotic Testing
Best Practices to Prevent Merge Conflicts with Copado 1 Platform
Software Bugs: The Three Causes of Programming Errors
How Does Copado Solve Release Readiness Roadblocks?
Why I Choose Copado Robotic Testing for my Test Automation
How to schedule a Function and Job Template in DevOps: A Step-by-Step Guide
Delivering Quality nCino Experiences with Automated Deployments and Testing
Best Practices Matter for Accelerated Salesforce Release Management
Maximize Your Code Quality, Security and performance with Copado Salesforce Code Analyzer
Upgrade Your Test Automation Game: The Benefits of Switching from Selenium to a More Advanced Platform
Three Takeaways From Copa Community Day
Cloud Native Applications: 5 Characteristics to Look for in the Right Tools
Using Salesforce nCino Architecture for Best Testing Results
How To Develop A Salesforce Testing Strategy For Your Enterprise
What Is Multi Cloud: Key Use Cases and Benefits for Enterprise Settings
5 Steps to Building a Salesforce Center of Excellence for Government Agencies
Salesforce UI testing: Benefits to Staying on Top of Updates
Benefits of UI Test Automation and Why You Should Care
Types of Salesforce Testing and When To Use Them
Copado + DataColada: Enabling CI/CD for Developers Across APAC
What is Salesforce API Testing and It Why Should Be Automated
Machine Learning Models: Adapting Data Patterns With Copado For AI Test Automation
Automated Testing Benefits: The Case For As Little Manual Testing As Possible
Beyond Selenium: Low Code Testing To Maximize Speed and Quality
UI Testing Best Practices: From Implementation to Automation
How Agile Test Automation Helps You Develop Better and Faster
Salesforce Test Cases: Knowing When to Test
DevOps Quality Assurance: Major Pitfalls and Challenges
11 Characteristics of Advanced Persistent Threats (APTs) That Set Them Apart
7 Key Compliance Regulations Relating to Data Storage
7 Ways Digital Transformation Consulting Revolutionizes Your Business
6 Top Cloud Security Trends
API Management Best Practices
Applying a Zero Trust Infrastructure in Kubernetes
Building a Data Pipeline Architecture Based on Best Practices Brings the Biggest Rewards
CI/CD Methodology vs. CI/CD Mentality: How to Meet Your Workflow Goals
DevOps to DevSecOps: How to Build Security into the Development Lifecycle
DevSecOps vs Agile: It’s Not Either/Or
How to Create a Digital Transformation Roadmap to Success
Infrastructure As Code: Overcome the Barriers to Effective Network Automation
Leveraging Compliance Automation Tools to Mitigate Risk
Moving Forward with These CI/CD Best Practices
Top 3 Data Compliance Challenges of Tomorrow and the Solutions You Need Today
Top 6 Cloud Security Management Policies and Procedures to Protect Your Business
What are the Benefits of Principle of Least Privilege (POLP) for My Organization?
You Can’t Measure What You Can’t See: Getting to know the 4 Metrics of Software Delivery Performance
How the Public Sector Can Continue to Accelerate Modernization
Building an Automated Test Framework to Streamline Deployments
How To Implement a Compliance Testing Methodology To Exceed Your Objectives
Cloud Security: Advantages and Disadvantages to Accessibility
Copado Collaborates with IBM to Accelerate Digital Transformation Projects on the Salesforce Platform
Continuous Quality: The missing link to DevOps maturity
Why Empowering Your Salesforce CoE is Essential for Maximizing ROI
Value Stream Management: The Future of DevOps at Scale is Here
Is Salesforce Development ‘One Size Fits All?’
The 3 Pillars of DevOps Value Stream Management
Gartner Recommends Companies Adopt Value Stream Delivery Platforms To Scale DevOps
The Admin's Quick Glossary for Understanding Salesforce DevOps
Top 10 Copado Features for #AwesomeAdmins
10 Secrets Management Tools to Facilitate Stronger Security Practices
5 Cloud Security Compliance Basics to Prevent Data Breaches
5 Data Security Management Fundamentals
Cloud Agnostic vs Cloud Native: Developing a Hybrid Approach
Making DIE Model Security vs. the CIA Security Triad Complementary, Not Competitive
The CI/CD Pipeline: Why Testing Is Required at Every Stage
DevSecOps Roadmap: From Architecture to Automation
Pets vs. Cattle: More Than an Analogy for Modern Infrastructures
Data Compliance Solutions Provide Greater Control Over Enterprise Data
Copado Cares: Free Training, Collaboration and Product Access for Global Response to COVID-19
12 Types of Social Engineering Attacks to Look Out For
Go back to resources
There is no previous posts
Go back to resources
There is no next posts
Ready to Transform Your Software Delivery Process?

Explore more about

No items found.