The short answerData scientists spend most of their time on data preparation rather than on model building, and that holds true for South African businesses too.
The short answer
Data scientists spend most of their time on data preparation rather than on model building, and that holds true for South African businesses too.
Why Data Preparation Matters More Than AI Tool Selection
Direct answer: Data scientists spend most of their time on data preparation rather than on model building, and that holds true for South African businesses too. We've found that companies rushing to pick an AI platform before assessing their data end up wasting months and a lot of money on rework.
Current as of 31 May 2026: This article has been reviewed for the 2026 South African AI, SEO, and automation market. Pricing, platform capabilities, Google rich-result rules, and AI model features change quickly, so verify live vendor documentation before procurement. For privacy and data handling, use the Protection of Personal Information Act as the baseline; for search and structured-data implementation, use Google Search Central.
About the figures in this article: The 1 to 5 scoring scale and the "don't start if" thresholds below are rules of thumb we use as a starting point, not published benchmarks. Adjust them to your own use case.
The uncomfortable truth? Your AI is only as good as the data feeding it. A sophisticated machine learning model trained on incomplete, inconsistent, or outdated data will produce confidently wrong answers. In our experience working with SA businesses, the single biggest predictor of AI project success isn't which tool you choose. It's how well you've prepared your data foundation.
TL;DR: Data preparation consumes a large share of AI project effort. Before selecting any AI tool, audit your data across six dimensions: completeness, accuracy, consistency, timeliness, accessibility, and format. South African businesses face unique challenges including spreadsheet-heavy processes, siloed departments, and POPIA compliance requirements.
What Are the 6 Dimensions of Data Readiness?
Poor data quality is expensive everywhere, and for South African SMEs even a small share of that waste can mean the difference between a profitable quarter and a loss. Data readiness isn't a single checkbox. It's six distinct dimensions that each need attention.
1. Completeness
Are all required fields populated? Missing values are the most common data problem we encounter. A customer database with many missing email addresses can't support an AI-driven marketing campaign. Check every table, every column.
What to look for:
- Blank fields in critical columns
- Records with only partial information
- Entire data categories that were never captured
2. Accuracy
Does the data reflect reality? Consider a logistics firm whose delivery addresses were long out of date: its route optimisation AI kept suggesting routes to buildings that no longer existed. Accuracy audits should compare a sample of records against real-world sources.
3. Consistency
Is the same thing recorded the same way everywhere? "Jhb", "Johannesburg", "JHB", and "Joburg" might all mean the same city, but your AI model treats them as four different locations. Inconsistent formatting is especially common when multiple people enter data manually.
4. Timeliness
How current is your data? Stale data creates stale predictions. If your sales data is only updated monthly, a demand forecasting model can't react to weekly trends. Real-time or near-real-time data feeds produce dramatically better AI outcomes.
5. Accessibility
Can your AI system actually reach the data it needs? We've seen South African businesses with excellent data locked inside legacy systems that have no API, no export function, and no way for modern tools to connect. Data sitting in someone's personal Excel file on a desktop PC isn't accessible data.
6. Format
Is the data machine-readable? PDFs, scanned documents, handwritten notes, and unstructured email threads all contain valuable information. But AI models need structured, tabular data to work effectively. Format conversion is often the most labour-intensive step.
Citation capsule: Poor data quality is a major, well-documented cost to businesses. The six dimensions of data readiness: completeness, accuracy, consistency, timeliness, accessibility, and format: each require separate assessment before any AI integration can succeed.
What Data Problems Do SA Businesses Commonly Face?
South African mid-market companies often lack a formal data governance framework. This creates a distinct set of challenges that differ from more digitally mature markets.
Spreadsheet-Heavy Processes
Many South African businesses still run critical operations on Excel. Payroll in one spreadsheet, inventory in another, customer records in a third. These files aren't version-controlled, they break when someone accidentally deletes a formula, and they can't scale. Before AI integration, spreadsheet-dependent processes need to migrate into proper databases or cloud platforms.
Siloed Departments
Finance uses Sage, sales uses a CRM, operations uses WhatsApp groups and paper forms. Sound familiar? When departments don't share data infrastructure, you end up with fragmented customer records, duplicate entries, and no single source of truth. AI needs a unified view.
Implementation note: We've assessed businesses where the same customer appeared three different ways across three systems: different spelling, different phone number format, different address. Reconciling these records is essential groundwork.
Manual Data Entry Errors
Human error is a fact of manual data entry, and every error adds up. Across thousands of entries per month, even a small error rate means hundreds of bad records. Those errors compound over time and poison any AI model trained on the data.
Load Shedding and Connectivity Gaps
South Africa's infrastructure challenges mean that cloud-based data collection sometimes fails. Offline-capable systems and local backups are essential. We've found that businesses without offline data capture strategies have significant gaps in their datasets.
How Do You Audit Your Data Estate?
Organisations that believe their data is ready for AI often overestimate their actual readiness. A structured audit prevents this overconfidence. Here's a practical approach that works for businesses of any size in South Africa.
Step 1: Inventory All Data Sources
List every place your business stores data. This includes ERP systems like SAP, Sage, or Pastel; CRM platforms; spreadsheets; email inboxes; paper records; WhatsApp conversations; and point-of-sale systems. Don't skip the informal ones. They often hold the most valuable insights.
Step 2: Map Data Flows
Document how data moves between systems. Where does a customer order start? Where does it end up? What happens in between? This mapping reveals gaps, bottlenecks, and duplication points.
Step 3: Score Each Source on the 6 Dimensions
Rate every data source from 1 to 5 on completeness, accuracy, consistency, timeliness, accessibility, and format. This creates a clear picture of where to focus your preparation effort.
Step 4: Prioritise by AI Use Case
Not all data needs to be perfect. If your first AI project is a customer support chatbot, prioritise your FAQ database and support ticket history. If it's demand forecasting, focus on sales and inventory data. Match your preparation effort to your planned use case.
Citation capsule: Organisations tend to overestimate their data readiness for AI. A structured four-step data audit: inventory, mapping, scoring, and prioritisation: prevents costly overconfidence and focuses preparation effort where it matters most.
What's the Difference Between Cleaning and Restructuring?
Data preparation can consume a large part of a data science team's time. But not all preparation work is the same. Understanding the difference between cleaning and restructuring helps you budget time and money accurately.
Data Cleaning
Cleaning fixes problems within your existing data structure. Think of it as repairing what you have.
Common cleaning tasks:
- Removing duplicate records
- Correcting spelling errors and formatting inconsistencies
- Filling in missing values where possible
- Fixing data type errors (text stored as numbers, dates in wrong format)
- Removing outliers that represent data entry mistakes
Effort: Moderate. The cost depends on volume and how many sources you have.
Data Restructuring
Restructuring changes how your data is organised. This is a bigger undertaking.
Common restructuring tasks:
- Migrating from spreadsheets to a proper database
- Merging siloed systems into a unified data warehouse
- Converting unstructured data (PDFs, emails) into structured formats
- Building APIs to connect legacy systems
- Creating master data management processes
Effort: Significant. The cost depends on complexity and the number of systems involved.
Smart AI Solutions insight: Many businesses assume they need a complete data restructuring project before starting with AI. That's not always true. Sometimes a focused cleaning effort on one specific dataset is enough to pilot your first AI use case, prove value, and justify the larger restructuring investment.
How Does POPIA Affect Data Preparation for AI?
South Africa's Protection of Personal Information Act (POPIA), enforced since July 2021, directly impacts how you prepare data for AI. The Information Regulator has signalled increasing enforcement activity, with administrative fines reaching up to R10 million for serious breaches (POPIA s109(2)(c)).
Consent and Purpose Limitation
POPIA requires that personal information be collected for a specific, stated purpose. If you collected email addresses for invoicing, you can't automatically use them to train an AI marketing model. Review the original consent scope for every dataset containing personal information.
Data Minimisation
Only collect and process data that's necessary for the stated purpose. When preparing data for AI, strip out personal identifiers that the model doesn't need. Does your demand forecasting model really need customer names? Probably not. Remove them.
Right to Correction and Deletion
Customers have the right to correct or delete their personal data. Your AI training pipeline needs to accommodate these requests. If a customer asks for deletion under POPIA, their data must be removed from training datasets too.
Cross-Border Data Transfer
If your AI platform processes data outside South Africa, POPIA's cross-border provisions apply. Ensure any cloud AI service you use either processes data locally or meets POPIA's adequacy requirements for international transfers.
Citation capsule: South Africa's POPIA legislation, enforced since July 2021, allows administrative fines of up to R10 million (s109(2)(c)). AI data preparation must address consent scope, data minimisation, deletion rights, and cross-border transfer rules before any personal information enters a model.
How Do You Build a Data Foundation That Supports AI?
Organisations with strong data foundations are better placed to meet their AI project goals. Building that foundation isn't a once-off project. It's an ongoing capability.
Centralise Where Practical
A data warehouse or data lakehouse gives AI models a single place to access information. For most South African SMEs, a cloud-based solution like Google BigQuery or Azure Synapse offers the right balance of cost and capability. Check current vendor pricing for your expected data volume.
Standardise Data Entry
Create clear rules for how data gets entered. Dropdown menus instead of free text. Validated phone number formats. Required fields enforced at the form level. Prevention is cheaper than cure.
Establish Data Ownership
Every dataset needs a named owner responsible for quality. Without ownership, data quality degrades silently. We've found that assigning data stewards to each department transforms data culture within six months.
Smart AI Solutions field note: When we assess a business, accessibility and format consistency usually score lowest on the six dimensions, while accuracy tends to score highest.
Invest in Integration
APIs, middleware, and integration platforms (like Zapier, Make, or custom connectors) ensure data flows between systems automatically. Manual re-entry is the enemy of data quality.
When Is Your Data "Good Enough" to Start?
Perfect data doesn't exist. Waiting for perfection means waiting forever. The practical question is: when is your data ready enough for your specific AI use case?
The 80% Rule
If your target dataset scores 4 or above on at least four of the six dimensions, and 3 or above on the remaining two, you're likely ready to pilot. This is a rule of thumb for roughly 80% readiness, which is sufficient for most initial AI projects.
Start Small, Learn Fast
Pick one well-defined use case with one well-understood dataset. A chatbot using your existing FAQ database. A reporting dashboard pulling from your CRM. A simple forecasting model using a year of clean sales data. Prove value, then expand.
What "Not Ready" Looks Like
Don't start if:
- A large share of critical fields are empty
- Data hasn't been updated for months
- No one can explain where the data comes from
- Multiple conflicting versions of the same dataset exist
- POPIA compliance hasn't been assessed
These are blockers, not speed bumps. Address them first.
We work remotely with businesses across South Africa, and on-site in Cape Town.
Frequently Asked Questions
How long does data preparation typically take?
The timeline depends heavily on how many data sources you have and how siloed your current systems are. Data cleaning is usually much quicker than restructuring, which is a major piece of work. Starting the audit early: even before you've chosen an AI tool: saves significant time.
Can we do data preparation ourselves or do we need external help?
Many cleaning tasks (removing duplicates, fixing formatting) can be handled internally with tools like Excel, Python, or OpenRefine. Restructuring and system integration typically benefit from external expertise, particularly when legacy systems are involved. A blended approach often works best.
What does data preparation cost in South Africa?
Costs depend on data volume, source count, and complexity. Basic data cleaning is much cheaper than full restructuring with system integration, so get a written quote after an audit. It's a significant investment, but it's also the foundation that determines whether your AI projects succeed or fail.
Does load shedding affect our data quality?
Yes. Power interruptions can cause data loss, incomplete transactions, and synchronisation failures between systems. Businesses should invest in UPS systems for critical data infrastructure and use cloud platforms with offline-capable data capture. We've assessed companies with months of gaps in their datasets directly attributable to load shedding outages.
How does POPIA change what data we can use for AI?
POPIA requires that personal data be used only for the purpose it was originally collected for. You need to review consent agreements, apply data minimisation principles, and ensure your AI processing meets the Act's conditions. Anonymous or aggregated data is generally lower-risk. Consult with a POPIA specialist if your AI use case involves customer personal information.
Conclusion
Data preparation isn't glamorous, but it's where AI projects are won or lost. Before you evaluate a single AI platform, assess your data across the six dimensions of readiness. Audit your sources, map your flows, and address the gaps that matter most for your intended use case.
For South African businesses, the path includes POPIA compliance, managing spreadsheet-to-database migrations, and bridging departmental silos. None of this is impossible. It just takes a structured, investigation-first approach.
Ready to assess your data foundation? Our Data Analytics and Reporting team partners with SA businesses to map data estates, score readiness, and create practical preparation roadmaps. So your AI investment delivers real results from day one.
Related Resources:



