Mass consumer service
Telco & Adtech
2020 - 2021Building a secure data vault for PII protection
Traditionally, such industries as banking, telcos, utility providers accumulate vast amounts of sensitive data due to their huge user base and sensitivity of their relationships with users.
In this case, a leading country-scale [REDACTED] provider planned to implement an in-house solution to store and process customers’ sensitive data in a compliant and secure way, and chose Acra database security suite to be its data security component.
Industry
B2B2C mass consumer services
Telco & Adtech platform
Technology stack
PostgreSQL
Java, Node.js, C++, Python
Linux + OpenShift
Regulations
GDPR
Regional privacy regulations
Challenges
Strong insider risks. [REDACTED] employs many personnel: developers, DBAs, internal cloud ops. A lot of people need access to the database and applications, yet should not have access to the actual sensitive data. Prior to this project, this requirement was achieved by the security team tightly controlling “who gets what” on a per-request basis, and trusted ops engineers with full access having to agree on each access case with security. The [REDACTED] understood that “manual access control” was exhausting and was looking to a centralized data security solution.
Compliance requirements. Security requirements and security policy mandate complex security configuration, where some columns/tables are encrypted, some are tokenized, access from some applications should return masked data.
Secure vault is actively used by BI teams and is part of ETL processes. BI teams should have access to the anonymised data, be able to pull large volumes of data quickly without impacting the performance for customers and internal teams.
Performance in key queries should be blazing fast. ETL-style OLAP queries over the secure personal data vault can be slow, so the performance delay introduced by the security module should not increase 10..15%.
Technology requirements
Access control for different types of consumers
The security system should provide a controlled, restricted and audited access for a number of different business applications with different security policies to the same database.
Support search queries on sensitive data without revealing it
The applications should be able to query data from the database using sensitive fields (SSN, MSISDN, unique IDs) without decrypting them.
Expose as an SQL database
Due to a multitude of customer’s applications using the data in the vault, exposing it directly as a PostgreSQL database was a strong requirement.
Our approach
Design and build a security layer around data
Focus on applying security policies to the data where the data is processed. Usage of application level encryption on data access points (DAO) allows to restrict the dataflow without re-engineering of applications.
Different tools for different workloads
Different Acra operating modes and components are suited for different workload types (OLAP ETL, OLTP, etc.). Picking optimal Acra components and configuration for each use case allows meeting non-functional requirements for each workload.
Match security policy for real-world security concerns
To avoid confusion, Acra’s security policy matches applications (data consumers) to the different trust tiers and, accordingly, matches necessary security controls to these tiers.
Solution
We have used Acra database security suite as the main security control for building a Secure Personal Data Vault.
In a nutshell, a security policy is very simple. Non-trusted applications (as identified by IP ranges and TLS certificate validation) get no access to sensitive data (e.g. encrypted data is not decrypted, tokens aren’t matched to the original fields, search returns no results), while trusted applications get appropriate level of access.
To enable secure data processing:
- Field level encryption of sensitive data that preserves the database structure and make it manageable, with sensitive fields inaccessible to DBAs in plaintext.
- Secure search over encrypted data using indexing of high-entropy PII fields: first and last names, SSNs, MSISDNs, etc.
- A combination of tokenization and encryption for low-entropy and easy-to-enumerate record types to prevent statistical attacks. It puts a non-sensitive and legitimately-looking token to the original table, encrypts the original data field and puts it to the separate table.
To enable secure BI:
- We have used a separate set of Acra replicas (Acra-OLAP) with a different configuration: decrypting the data, then masking out the sensitive fields, passing data to BI ETL so that BI system is not aware of the original content of sensitive fields.
- Since BI is mostly complex OLAP traffic, we used AcraCensor’s logging capability to highlight suspiciously looking substrings in queries for further investigation.
To enable replication and backups:
- The customer performed replication and backups as usual, Acra ensured that sensitive data stays encrypted.
To enable defense-in-depth:
- Row-level security in PostgreSQL.
- Audit log with filtered query values (as they may be sensitive) recorded using Acra’s Censor & Log modules to the offline log.
- Extensive security monitoring that relies on Acra and surrounding services to trigger security alerts.
Products and services involved
Acra, a database security suite
Acra is a set of data security modules: encryption, masking, tokenization, query filtering, logging. Due to its modular nature, Acra fits complicated architectures and use cases without introducing a significant performance penalty.
Read moreAcra, a database security suite
Security engineering & architecture
Cossack Labs security engineering services help development teams to spend less time on irrelevant stuff, focusing on pragmatic security, actual infrastructure and product.
Read moreSecurity engineering & architectureBenefits
The Customer was planning to implement a sophisticated set of data security controls themselves. Instead, delivering Acra-based solution:
- Took 3 months to design, build and deploy (instead of 12 months budgeted for a homebrewed solution).
- Developer budget was 220 hours instead of 3800 budgeted for a homebrewed solution.
- Provided a security team with convenient means to protect sensitive fields and control database queries without making developers’ life harder.
Results and outcomes
[REDACTED] ended up having a solution that satisfies complex security requirements and enables a single point of control for PII data protection policies.
It led to unblocking of development and operational processes (as access to the production environment became available without passing security controls every time) and improved usage of sensitive data in various customer’s business applications in a compliant way.
From a technical perspective, Acra centralised and unified data security measures, separated data flows for apps and the BI team, and satisfied performance requirements for OLAP queries.
Integrating data protection into massive consumer applications
We help you focus on serving your customers better, while relieving your team from security engineering pains and making your users confident that their data is safe with you.
