API penetration testing methodology

Introduction to API

API (Application Programming Interface) is an interface designed to help programs, devices, clouds and their databases interact and integrate. It connects hundreds of apps and makes them all works as the one. In case of user interface where a person interacts with an app via buttons that are associated with the app’s functions, application programming interface is all about lines of code from one app interacting with lines of code from another app to exchange data using various programming languages. Nowadays, in the Internet of Things era and exponentially increasing of Data, nearly 90% of developers use APIs in many fields around the world.  These are the most popular APIs’ protocols: RPC (upgraded to gRPC), SOAP, REST (or RESTful), GraphQL.

Each protocol has various of specifications which describe it from different sides of usage and needs. For SOAP protocol it’s SOAP specification (how to define SOAP protocol), WSDL (how to describe SOAP-service), WS-Security (how to secure SOAP-based services) etc. In REST case that in 99% uses HTTP protocol for communication,  specification also will be HTTP-based. Another specification for REST is Swagger that describe, product and visualize RESTful web services and OpenAPI that based on Swagger 2.0 specification.

Specification contains documentation (how to use it) and a deeper explanation (how it works). Documentation is about how to structure requests to another system so that this system understands it’s queries and responds to them.

One of the most common API types is Web API or web service or RESTful service that sends messages over HTTP protocol with CRUD operations — GET / POST / PUT / DELETE that makes it possible to communicate between web services even if they are built using other languages or platforms. SOAP can use HTTP protocol too, but, in this case, you can get Response only in XML format. This makes it heavier and less flexible compared to REST, which gives you various formats — XML, JSON, Atom, RSS, CSV, HTML, and others.

Why we need GraphQL at all? REST can over-fetch data. Imagine that a client should send requests to different endpoints (resources) and collect, filter, and compose all the data somehow. The request is processed and the client gets their data, but on top of that, they get lots of unrelated data from the source just because REST is designed this way. To deal with this issue, GraphQL was created. It is designed to work with one endpoint and give users the ability to get the exact data they need — no more, no less (see our GraphQL methodology for more info).

Key features

Function API calls — it’s language statements that allow to make requests on the server, composed with verbs and nouns.

Endpoints — endpoints of interactions — different options that you can use in API. These are actual address of the resource.

API key — allows you to interact with API and prevent the server from overloading and abuse.

API Gateway — when a user sends an HTTP request, API Gateway checks Parameter Validation, Rate limiter, Allow / Deny list, conducts an

Authentication / Authorization check (Identity Provider), etc.

API Authentication implemented with JSON Web Token (JWT) and OAuth.

REST API:

• gives limited access for another system — borders other apps can’t cross.
• uses HTTP caching.
• has uniform interface — one naming convention, data format, and endpoint format.
• has session independence and doesn’t store info connected to the previous session.
• is system-independent, which means that system transformations won’t affect API functions.

Web API communication is based on the client-server architecture as the World Web!

Testing APIs with OpenAPI specification leads to check if every response and request and data structure in them, client libraries, server code, JSON schema validation etc. (in YAML or JSON file that included all of them aka API schema) are meet the OpenAPI standards and definitions. To make schema human-readable use Swagger UI.

Note! A company chooses API that fits its business logic and needs best, not because it’s the most popular one.

Instruments to use

Postman — send Requests and receive Responses in the nice and easy to understand format.

Swagger UI and SoapUI — visualize the API logic and interact with it.

See Swagger API Testing also.

Kiterunner — API scanner for endpoints and content discovery.

BurpSuite and ZAP — Intercept the requests and responses from API.

unfurl — extracting paths from URL lists.

FFUF — tool for API fuzzing.

ILSpy — decompile API projects’ files (for white-box).

MindAPI — Reconnaissance and testing tips.

Checklist — follow this checklist during pentest.

API key guesser — guess the API key.

OpenApi.Tools — lots of useful tools for OpenAPI.

Reconnaissance phase

Enumerate all possible endpoints locally in the docs (in WADL /WSDL format) or in the web and build logic “tree view” in Burp Suite or Swagger UI. Documentation uses WSDL / WADL format and usually saved in the ?wsdl or /application.wadl or /application.wadl?detail=true in path like https://api.ex/api/?wsdl. Compare the local and public API docs seeking for hidden functions, methods or endpoints. Use json2paths tool for endpoint enumeration. For defining specifications (that often more useful than documentation) swagger endpoints use this. You can also bruteforcing paths with Burp Intruder and using wordlists or create specific one after exploring your target.

Port scanning, enumerate services running on the server, enumerate IP addresses.

Use SOAPUI or WSDLer Burp Suite Extension for SOAP.

Use Postman for REST for parsing docs.

Discover HTTP Methods (GET, POST, PATCH. PUT, DELETE).

Discover the API version in the URL and possible usage of older versions in login / register / user paths and endpoints. Use V1 or V2 or V3 etc. in the URL paths. The older versions are usually more vulnerable.

Note that API for developers is not the API for users, so test them separately. Discover the API style and documentation, define if there’s only one API architecture (SOAP + REST or just REST / SOAP). For this purpose change the content-type to “application/xml”, add a simple XML in the request body, and see how the API response to it.

Discover the less useable features in the paths to check for interesting vulnerabilities. Developers usually put much more efforts to the main APIs features.

Find out the subdomains and sibling domains to find previous API versions and stages. Scan java scripts files for URLs and understanding API infrastructure. Use online web-archives.

Discover the API Implementation (Open source on GitHub, products, custom API implementation).

Authentication and Authorization. Find all possible endpoints which requires authentication and other publicly accessible. There’s always more than one way to authenticate to an API. Modify the URL paths with key words and test all the URLs.

Find auth methods (cookie based or header based). Identify handlers (JWT (JSON Web Token), Encrypted value, value to save the user’s state, encoded / encrypted serialized value, hashed user value – hashed username, user ID)

Use Kiterunner to scan the endpoints and unfurl to extract paths from URL lists. This will help in generating custom wordlists phase.

Broken Object Level Authorization

Simply manipulate with ID in the URL endpoint by replacing it with another and get access to the sensitive data that you shouldn’t see and enumerate. Authorize to your account and then replace your ID value in the URL or Headers (e.g., X-User-Id) providing the same authorization data. Then try to get unauthorized access to the other user’s accounts or documents and even manipulate with them, which is leads to data disclosure, data loss, account takeover. Real world example: T-Mobile in 2017 Attacker discovered that phone number were used as user ID in the URL.

Tools: Astra / apidor / AuthMatrix / Autorize / Auth analyzer / Susanoo

Note! Secure REST services must only provide HTTPS endpoints. This protects authentication credentials in transit, for example, passwords, API keys, or JSON Web Tokens. It also allows clients to authenticate the service and guarantees the integrity of the transmitted data.

Broken User Authentication

If API doesn’t have rate limiting for requests, captcha, account lockout, weak login/password policy and cryptographic mechanism there’s risk of registered usernames enumeration. Try to use trivial and custom usernames and passwords in the login page, brute force passwords or 2FA codes. See if passwords, usernames and session tokens use in the URL paths. Check the session token validation / signing and try to bypass it. This leads to account takeover and perform sensitive actions from the other accounts like money transferring.

To bypass authentication, you can modify JSON parameters if authentication mechanism works only with plain strings. Instead of {“id”:111} use {“id”:[111]} etc.

Lack of Resources & Rate Limiting

Try to overload the API server with huge number of requests that of course consume resources like network, memory storage and processor power. For this purpose, no authentication is required. Try to send long values / payloads in the request parameters, upload heavy files many times, increase the speed of requests etc. This leads to Denial-of-Service thread or makes service unavailable.

Broken Function Level Authorization

Discover the hidden endpoints you as a user shouldn’t have access to with changing HTTP method from GET to POST / PUT etc., changing users’ parameters in the URL to escalate vertical or horizontal access privileges. The main task is to find administrative functions, get access to them and manipulate with their sensitive data. Try to change the user roles, register / delete / enumerate users. So, broken level authorization and user hierarchy leads to manipulate with administrative / sensitive data.

Mass Assignment

Look up for properties you shouldn’t see as a user in the response body by changing POST to GET in the request. After that you may see additional info about the user account. For example, {“user_is_admin” : false} or {“email_verifed”: false}. If so, use this property with the POST / PUT header in the new request and overwrite the value. Exposure of internal objects and misconfigured whitelisting of internal properties and code variables leads to overwriting sensitive data, privilege escalation, bypassing security mechanisms etc.

Improper Assets Management

On the Recon phase find out if there’s old, unpatched API versions and documentations running on some hosts that connected to the same databases. To figure out it, see if docs were updated, host inventory is outdated, old versions are running on the server without patching, API running in production, test, dev stage. Simply change e.g., V2 to V1 in the endpoints and see whether you able to expose sensitive information, gain access to developers’ data or technology that previously wasn’t implemented with security features etc.

Excessive Data Exposure

Intercept the HTTP traffic via Burp Suite or ZAP and look up for sensitive data in Response body and error messages. Data exposure in Response is designed to be there and it’s up to developers to implement addition filtering in order to secure data. Misconfigured data filtering leads to exposure of sensitive data. Try to enumerate all the API endpoints and sniff the traffic from them.

Injection by design & Export Injection

Replace the parameters in the requests with SQL  / NoSQL / XML / OS command / ORM / ODM payloads after exploring the source code or scan and fuzz the endpoints. Use Payloads All The Things. If client data is not sanitized, filtered or validated properly there’s high risk of Injection attack and DoS, data loss or overwriting etc.

While exporting PDF file from the source there’s a chance to usage of external libraries for converting HTML. Simply inject HTML tags (iframes tags) into content headers in the HTTP request: <h1>Hello<h1><iframe src = “C:/…”. This can lead to export injection. For more details check this.

Security Misconfiguration

Discover the common security issues aka systems are out of date, missing of TLS or HTTP is not secure in some endpoints, lack of Security Headers, unnecessary HTTP headers are enabled, missing of CORS (especially if API designed to be accessed from browser-based clients), sensitive data exposure in the error messages. Try to exploit findings for expose sensitive data and even compromise the whole server. Search for an unprotected files, unnecessary services and paths to gain unauthorized access or simply knowledge about how the system works and what services and its versions implemented in the API server.

Extensions for Burp Suite API testing

OpenAPI Parser

is an extension aimed at streamlining the process of performing web service assessments involving OpenAPI based APIs. https://portswigger.net/bappstore/6bf7574b632847faaaa4eb5e42f1757c

burp-rest-api

default tool for BurpSuite-powered web scanning automation.

https://github.com/vmware/burp-rest-api

BurpKit

Is a BurpSuite plugin which helps in assessing complex web apps that render the contents of their pages dynamically.

https://github.com/allfro/BurpKit

swurg

designed for OpenAPI testing.

https://github.com/AresS31/swurg

burp-vulners-scanner

search fingerprints in HTTP response and check found version in vulners.com vulnerability database.

https://github.com/vulnersCom/burp-vulners-scanner

Freddy, Deserialization Bug Finder

helps with detecting and exploiting serialization libraries/APIs.

https://portswigger.net/bappstore/ae1cce0c6d6c47528b4af35faebc3ab3

sqlipy

is a Python plugin for Burp Suite that integrates SQLMap using the SQLMap API.

https://github.com/codewatchorg/sqlipy

Peach API Security

is an automated security testing solution that allows organizations to test their web APIs against the OWASP Top-10 and PCI Section 6.5. https://github.com/PeachTech/peachapisec-burp

Setting up the testing environment on KALI LINUX

VAmPI https://github.com/erev0s/VAmPI

This is the testing environment (website which is hosted locally) for practicing pentester’s skills and testing API security/scans/pentest tools. Have fun! Make sure that all requirements for app are satisfied.

1. Clone the repository:

git clone https://github.com/erev0s/VAmPI

2. Open the folder with app cd VAmPI and run:

sudo docker build -t vampi_docker:latest .

docker run -d -p 5000:5000 vampi_docker:latest

3. Open the browser and navigate to http://localhost:5000 or http://localhost:5000/books/v1 to see books info.

vulnerable-api https://github.com/rahulunair/vulnerable-api

This repository contains an example Python API that is vulnerable to several different web API attacks.

1. Clone the repository:

git clone https://github.com/rahulunair/vulnerable-api

2. Open the folder with app cd vulnerable-api and run:

sudo docker run -tid -p 8081:8081 --name api mkam/vulnerable-api-demo

3. Open the browser and navigate to http://localhost:8081

Tiredful-API https://github.com/payatu/Tiredful-API

Tiredful API is intentionally designed broken app. The aim of this web app is to teach developers, QA or security professionals about flaws present in webservices (REST API) due to insecure coding practice.

1. Clone the repository:

git clone https://github.com/payatu/Tiredful-API

2. Open the folder with app cd Tiredful-API and run:

sudo docker build -t tiredful .

sudo docker run -p 8000:8000 --name tiredful -it tiredful

3. Open the browser and navigate to http://localhost:8000

crAPI – Completely ridiculous API https://github.com/OWASP/crAPI

Completely ridiculous API (crAPI) will help you to understand the ten most critical API security risks. crAPI is vulnerable by design, but you’ll be able to safely run it to educate/train yourself.

1. Clone the repository:

git clone https://github.com/OWASP/crAPI

2. Open the folder with app cd Tiredful-API and run:

curl -o docker-compose.yml https://raw.githubusercontent.com/OWASP/crAPI/main/deploy/docker/docker- compose.yml

sudo docker-compose pull

sudo docker-compose -f docker-compose.yml --compatibility up -d

3. Open the browser and navigate to http://localhost:8888

vAPI – Vulnerable Adversely Programmed Interface https://github.com/roottusk/vapi

vAPI is Vulnerable Adversely Programmed Interface which is Self-Hostable API that mimics OWASP API Top 10 scenarios in the means of Exercises.

1. Clone the repository:

git clone https://github.com/roottusk/vapi

2. Open the folder with app cd Tiredful-API and run:

sudo docker-compose up -d

3. Open the browser and navigate to http://localhost/vapi/

API security tools

FFUF https://github.com/ffuf/ffuf

API endpoint, paths, subdomains enumeration.

You’ll need the wordlist :  https://github.com/danielmiessler/SecLists. Website for testing http://ffuf.me/

1. Installation:

git clone https://github.com/ffuf/ffuf && cd ffuf && go get && go build

2. Using common.txt wordlist with GET header for 200 OK response with results output as a file in json format. -recursion  tells ffuf that if it enounters a directory it should start another scan within that directory and so on until no more results are found.

ffuf -w wordlist/Discovery/Web-Content/common.txt -recursion -X GET -mc 200 -o results.json -u http://ffuf.me/cd/basic/FUZZ

Now you can see results in the results.json file.

Swagger-EZ https://github.com/RhinoSecurityLabs/Swagger-EZ

A tool geared towards pentesting APIs using OpenAPI definitions for building sitemap and enumerate the API endpoints for Swagger (OpenAPI) APIs.

1. Installation:

git clone https://github.com/RhinoSecurityLabs/Swagger-EZ

2. Open Burp Suite. Open file index.html in Burp Suite Browser to Intercept the API requests and build a site map. Click on Load -> Auto Fill -> Send All.

After Send All now you can see in the Burp Site map the tree for example target API petstore.

Astra https://github.com/flipkart-incubator/Astra

API scanner for common vulnerabilities aka SQL injection, CSS, Information Leakage, Broken Authentication and session management, CSRF, CORS, JWT attack etc.

1. Clone the repository and run:

git clone https://github.com/flipkart-incubator/Astra.git

cd Astra

sudo docker build -t astra .

sudo docker run --rm -it --link astra-mongo:mongo -p 8094:8094 astra

Open the http://localhost:8094 in browser and run the scan.

Kiterunner https://github.com/assetnote/kiterunner

API scanner that allows you to bruteforcing the API endpoints with wordlists.

Clone the repository:

git clone https://github.com/assetnote/kiterunner

Download files from the repository. Extract them in the Kiterunner folder:

Open terminal in the Kiterunner folder and run:

make build

sudo ln -s $(pwd)/dist/kr /usr/local/bin/kr

kr kb compile routes-large.json routes-large.kite

Now we can use wordlist from the Kiterunner wordlist list. To see the list run:

kr wordlist list

In our example we use raft-small-files-lowercase wordlist. Run the scan:

kr scan http://localhost:8000 -A raft-small-files-lowercase

Other API security tools

ZAP (fuzzer) https://www.zaproxy.org/

Dredd https://dredd.org/en/latest/

CATS https://github.com/Endava/cats

Zerocode https://github.com/authorjapps/zerocode

Susanoo https://github.com/ant4g0nist/Susanoo

WapiFuzz https://github.com/ysoftdevs/wapifuzz

OpenAPI Vulnerability Scanner https://github.com/s3rgeym/openapi-vulnerability-scanner

Apidor https://github.com/bm402/apidor

Conclusion

APIs systems are susceptible to a broad range of vulnerabilities. Understanding potential dangers is crucial in order to take the steps necessary to efficient pentest an application.

Cybersecurity doesn’t have to be complicated. With CQR company, you get a team of professionals to perform a complete security check of your app, whether it runs on REST, GraphQL, or any other API. Schedule a free demo today or learn more about our workflow.

While we covered some of the most common vulnerabilities, there are many other dangers out there. With time some vulnerabilities are addressed by developers while new vulnerabilities are being found and exploited, attacks get more sophisticated, and hackers more creative.

Addition information for study

REST Security Cheat Sheet

https://cheatsheetseries.owasp.org/cheatsheets/REST_Security_Cheat_Sheet.html

Awesome-Application-Security-Checklist

https://github.com/MahdiMashrur/Awesome-Application-Security-Checklist

Microsoft REST API Guidelines

https://github.com/microsoft/api-guidelines

HolyTips API Security Checklist

https://github.com/HolyBugx/HolyTips/blob/main/Checklist/API%20Security.pdf

API Security Empire

https://github.com/Cyber-Guy1/API-SecurityEmpire

API-Security-Checklist

https://github.com/shieldfy/API-Security-Checklist

Public APIs

https://github.com/public-apis/public-apis

Awesome-api-security

https://github.com/arainho/awesome-api-security

Vulnerable public web-service with API

https://hack-yourself-first.com/

SOAP vs REST. What’s the Difference?

https://smartbear.com/blog/soap-vs-rest-whats-the-difference/

BOLA, IDOR, MA, BFLA. Welcome to the OWASP API Top 10!

https://www.youtube.com/watch?v=BdzxmuQhn_M