15 Practices for Shielding Your APIs from Attack: #3 – Concepts, Cryptography, & Credentials
Published: June 4, 2023
This post shares experiences from our multi-year journey working with a Canadian client to help them improve their API security and security management. In our API gateway project, there are several architectural and design-related principles and ideas that we either adopted or invented, to ensure increased security. This is a long post, so we start with an overview of the practices to let you jump around if you don’t wish to read the whole post.
Overview
In summary, the ideas, architecture, and design principles important to API and product security are:
- CI/CD pipelines.
- APIs and interfaces to make credential management automatable or easy to perform securely (automated to rotate credentials).
- Fully automated system validation (comprehensive product coverage) through the unit and functional testing.
- There will be more detail in the future about implementing this idea, and the technologies that make it easier.
- Prevent accidental API exposure through badly configured deployments.
- Encourage and enforce the use of approved security and deployment patterns, like ensuring basic authentication and logging.
- Defend specific APIs at a greater level of security.
- Work with the Identity Systems and provide OAuth2 as a service to APIs.
- Spot security implementation mistakes quicker.
- Avoid implementing unnecessarily complex software.
- Keep designs simple and fit for purpose (design hygiene).
Network and Credential Encryption Practices
Simple network security approaches are key. Always insist on the basics like using HTTPS everywhere, and locking down ports by default (deny-by-default, only open ports as needed, and ideally only open endpoints within private subnets). We also focus on common credential ‘hygiene’, such as storing credentials in an encrypted format, and providing interfaces so customers can self-serve changes to their API access keys and credentials. It is paramount to avoid transmitting API secrets to API customers over insecure channels such as email. This practice also keeps us on the right side in a ‘non-repudiation’ situation, whereby only clients have knowledge of their credentials and are responsible for their security. This restricted access does, of course, require an interface where a specific customer is authorized to manage a particular application profile, but others are not. Whether a security setting is changed via an API or the management portal, we track ALL security events in detail, including who changed what, when, and from where. These details enable future investigations into activities like credential change or management. We’ll cover more in a future post about Observability & SEIM.
All these concepts are kept in a simple checklist, maintained by the team, to review when they decide to make architectural or software changes. Other security agreements and team practices are captured in other checklists and are a working agreement between the members of the DevSecOps team.
Rotate, Repave, Repair – Automation & APIs
Another practice captured and encoded into our build and deployment pipelines are the concepts of Rotate, Repave, and Repair (the Three R’s of security). Wherever possible and practical, the product team ensures that their pipeline pulls the latest versions of containers or libraries into the product build process. Even if a firmware, library, or platform is a manual step and decision, the pipeline and fully automated tests enable the team to validate the new change (e.g. a library replacement), with confidence that no defects have been introduced with the update. Staying current on the latest versions of dependencies helps to close security gaps because older code tends to be more exploitable (not just entropy at work, but also attackers have had more time with the older code to discover and exploit gaps that were there all along).
Automated infrastructure deployment pipelines allow the team to rebuild the entire product from the ground up from secure components each time, which means that steps such as opening certain ports on servers are performed repeatably by the CI/CD pipeline.
In one example, this automation helped us to quickly recover from a large threat/defect identified in a load balancer product completely out of the blue. A popular infrastructure vendor announced a critical security flaw in their system. Our team was able to redeploy that infrastructure on the latest (fixed) version quickly and validate its functionality, prior to swapping out the component at risk. This swift response was only possible because we automated our deployment, set-up, and testing. The result? Speed, security, reliability, and no impact on customers – hallmarks of our team as a result of these practices.
We will talk more about rotating credentials in the next post, but all the components in our system use our own customer-facing APIs for credential rotation to ensure we are ‘eating our own caviar’. Indeed, this means we can tell our customers that we too rotate key credentials on a daily or weekly basis, all without any manual intervention, and without anyone ever seeing the credentials used by the software.
Policy Sets: Security Patterns Pre-Encoded
Our API gateway introduced a specific deployment pattern for APIs in production that our team invented. Called ‘policy sets’, we have designed the workflow for the day-to-day operations team responsible for deploying customers’ services on the gateway so that they select from security patterns pre-encoded in sets of policies.
Using policy sets means that it is practically impossible for someone on this team, regardless of skill level or experience, to configure and deploy an API without at least basic levels of authentication and security logging. Other policies can be selected from the set or are required to be configured in the set, depending on the pattern the policy set represents.
Identity, OAuth2 and the Resource Server
One of these policies and policy sets is responsible for enforcing OAuth2 security. Solsys had previously worked with our Canadian customer around Identity and Access Management (IAM) to establish an Identity Platform (IdP), which hosts their customer’s identity and can issue OAuth2 tokens expressing the customer’s authorized relationship to their products and services.
The API gateway provides OAuth2 as a service to downstream APIs (without API developers having to validate customer tokens in their physical API) and also works with the IdP to enable this policy, when needed, to play the full role of the Resource Server in the OAuth2 model. As a result, our gateway can be configured, on an API and method-by-method basis, to enforce account or product validation for a specific customer.
What this means is that any client accessing an important API protected this way has to provide an OAuth2 token, generated by customer authorization, that the gateway validates to ensure that the client is only accessing accounts or products for which the customer has authorized access. For example, accessing an account balance by billing account number can require the token to represent a customer who has that billing account associated with their identity profile.
This extra level of checking on the gateway protects the APIs from accidental mistaken access, but also malicious use of software clients’ credentials should their API key become compromised.
Kong, as an API gateway and critical partner, provides the framework for fully orchestratable and customizable policies. We were able to use this capability to adapt OAuth2 policies to perform the specific validations required and overlay our policy set concept, discussed above.
We will talk more about the Resource Server and OAuth2 security concepts when we cover how these systems protect against specific attacks in the later post about specific attack vectors identified by OWASPs top 10.
eXtreme Programming Concepts
Lastly (for this post), concepts from the eXtreme Programming (XP) movement play a prominent role in software security in general.
If you’re not familiar with XP, many of these ideas pre-date Scrum and the Agile Manifesto. Most of the concepts in XP are programming practices and concepts, whereas in general terms Agile concepts can apply to all kinds of complex work. Going into depth here is out of the scope of this series, and our teams like most, follow XP practices to varying degrees. Some ideas in XP however can really help to ensure software security.
For example, Pair Programming, the idea of two developers working together to produce code, can help a team ensure that common development mistakes are caught immediately. It also helps teams to be confident that everyone’s code is adhering to agreed working practices – including those related to security. Pair Programming tends to be more comprehensive and likely to catch issues than post-development code reviews (although these aren’t mutually exclusive practices).
The benefits of simplicity in code, another XP idea – and its impacts on security – cannot be overstated. Building the least amount of code to meet the need is a great way to avoid building software that contains security vulnerabilities. Code that isn’t there, can’t have a security defect! Our team operationalized this principle during joint design sessions, Pairing, and code reviews. Complexity in code is where we often find unintended behaviours and security holes. Removing unused code, continuous refactoring (boy scout rule as it’s sometimes called), and team-agreed coding standards all help towards this goal. Our teams also agree to measure some of these coding standards through code analysis tools, which will be discussed more in a later post.
The concept of simplicity in code can also be reflected in general design hygiene. The idea of keeping designs minimal (simple), but also considering whether a function should even be there or not, is important. For example, having considered RESTful design for any implemented APIs means that the team is more likely to implement an API correctly. There will be fewer ‘strange gaps’ in terms of how the API works, operations aren’t just left in because they might be needed, rather they are built only with intention. Following good design principles also means that your software will make appropriate use of the functions and patterns in an API implementation library instead of working against them. When we use the tools we select for their intended purpose, we’re more likely to benefit from their functionality (that might protect us from security problems), rather than introduce unexpected behaviours (which can lead to new security problems).
What’s Next?
In the next post in the series, we will talk about some of the tools, systems, and platforms that helped our team improve and enforce security during our work with our Canadian customers, and reduce the security risk posed by APIs.
Solsys would be pleased to share more details of this experience with you. We have various workshops, such as our API attack vector workshop, and other services available to support you in your API & Software security journey. Please reach out to us by using the contact form on this site.
