Certified Python Data Analyst Professional

The big picture

Artificial Intelligence (AI) is the broad goal of making machines do things that normally need human intelligence. Machine Learning (ML) is the main way we reach that goal today: instead of writing rules by hand, we let a program learn patterns from data. Deep Learning (DL) is a powerful branch of ML that uses neural networks with many layers. Data Science is the wider craft of turning raw data into decisions — it uses statistics, programming, and ML together.

Three kinds of machine learning

• Supervised learning — you have labeled examples (input → known answer) and the model learns to predict the answer. E.g. predict a house price from its features.
• Unsupervised learning — no labels; the model finds structure on its own, such as grouping similar customers (clustering).
• Reinforcement learning — an agent learns by trial and reward, like a program learning to play a game.

The data-science workflow

Almost every project follows the same loop. Knowing it keeps you oriented no matter how complex the problem gets:

• 1. Define the question and success metric with stakeholders.
• 2. Collect and store the data (files, databases, APIs).
• 3. Clean and explore it (EDA) — understand what you actually have.
• 4. Prepare features and split into train/test sets.
• 5. Train models, evaluate, and tune them.
• 6. Deploy the chosen model and monitor it in production.

Where the tools fit

In this program you will use Python as the language, SQL to pull data from databases, NumPy/Pandas to explore it, Matplotlib/Plotly to visualize it, scikit-learn to build models, and Streamlit to ship them. Each later topic in this guide is one stop on that path.

Why Python

Python reads almost like English, runs everywhere, and has the richest data ecosystem (NumPy, Pandas, scikit-learn, TensorFlow). You do not need to be a software engineer — you need fluency in a focused set of building blocks.

Variables and core data types

A variable is a name that points to a value. Python figures out the type for you. The everyday types are numbers (int, float), text (str), booleans (True/False), and the containers below.

price = 250.0            # float
name  = "Cairo villa"    # str
is_sold = False          # bool

nums   = [1, 2, 3, 4]                 # list  — ordered, changeable
point  = (30.0, 31.2)                 # tuple — ordered, fixed
unique = {"cat", "dog"}               # set   — no duplicates
person = {"name": "Sara", "age": 29}  # dict  — key → value

Control flow: decisions and loops

if/elif/else choose between paths; for loops repeat over a collection. Indentation (4 spaces) is how Python groups code — there are no braces.

for n in nums:
    if n % 2 == 0:
        print(n, "is even")
    else:
        print(n, "is odd")

Functions — package logic you reuse

A function takes inputs, does work, and returns a result. Writing functions keeps analysis clean and repeatable instead of copy-pasting code.

def price_per_m2(price, area):
    """Return price per square meter."""
    return price / area

print(price_per_m2(250000, 120))   # 2083.33

List comprehensions & libraries

A list comprehension builds a new list in one readable line — a Python idiom you will see everywhere. And `import` brings in libraries: this is how NumPy and Pandas (next topics) enter your notebook.

squares = [n * n for n in range(5)]   # [0, 1, 4, 9, 16]

import pandas as pd
df = pd.read_csv("sales.csv")         # your data, one line away

Tables, rows, columns

A relational database stores data in tables — a grid of rows (records) and columns (fields). Each table usually has a primary key that uniquely identifies a row, and tables relate to each other through those keys. MySQL is the engine you will use.

The anatomy of a query

Reading data is one statement whose clauses always run in a fixed logical order. Learn this skeleton and you can read almost any query.

SELECT   city, COUNT(*) AS deals, AVG(price) AS avg_price
FROM     sales
WHERE    year = 2025
GROUP BY city
HAVING   COUNT(*) > 10
ORDER BY avg_price DESC
LIMIT    5;

JOINs — combining tables

Data you need is usually spread across tables (customers, orders, products). A JOIN stitches them back together on a shared key. INNER JOIN keeps only matches; LEFT JOIN keeps every row from the left table even when the right has no match.

SELECT o.id, c.name, o.total
FROM   orders o
JOIN   customers c ON c.id = o.customer_id
WHERE  o.total > 1000;

Aggregation is where insight lives

• COUNT, SUM, AVG, MIN, MAX summarize many rows into one number.
• GROUP BY computes those summaries per category (per city, per month).
• This single pattern answers most business questions: 'what is the average/total X per Y?'

Descriptive statistics: center and spread

First, summarize a column. The center is the mean (average) or median (middle value — robust to outliers). The spread is the standard deviation: small means values hug the mean; large means they scatter. Always look at both — the same mean can hide very different data.

Distributions & the normal curve

A distribution shows how often each value occurs. The famous normal (bell) curve appears everywhere. Its rule of thumb: about 68% of values fall within 1 standard deviation of the mean, 95% within 2, and 99.7% within 3. That is how you judge whether a value is 'unusual'.

Probability in one minute

• Probability measures how likely an event is, from 0 (never) to 1 (certain).
• Conditional probability P(A|B) — the chance of A given B already happened — is the heart of many models (e.g. spam filters).
• Independent events multiply: P(A and B) = P(A)·P(B) when one does not affect the other.

Inference: is this pattern real?

You rarely have all the data — only a sample. Inferential statistics lets you generalize from the sample to the population. A hypothesis test asks: 'if there were truly no effect, how surprising is what I saw?' The p-value answers it: a small p-value (< 0.05 by convention) means the result is unlikely to be pure chance.

Correlation ≠ causation

Correlation measures how two variables move together (from -1 to +1). It is essential for feature selection — but two things can correlate without one causing the other (ice-cream sales and drownings both rise in summer). Proving causation needs a controlled experiment, not just a correlation.

Vectors and matrices = your data

A single example (one house: its size, rooms, age) is a vector — a list of numbers. Stack many examples and you get a matrix: rows are examples, columns are features. Every dataset you load into Pandas is, underneath, a matrix — which is why linear algebra is the language of ML.

The dot product — the workhorse

Multiply two vectors element-by-element and add the results: that single operation is how a model combines features with weights to make a prediction. A linear model is literally weights · features + bias.

import numpy as np
weights  = np.array([0.4, 0.3, -0.1])
features = np.array([120, 3, 15])       # size, rooms, age
prediction = weights @ features + 50    # @ is the dot product
# 0.4*120 + 0.3*3 + -0.1*15 + 50 = 97.4

Derivatives = the direction of improvement

A derivative measures how a function changes as you nudge its input — its slope. In ML, the function is the error, and the derivative (gradient) points uphill toward more error. So we step the opposite way to reduce it. That is gradient descent: the single algorithm behind training linear models, neural networks, and most of deep learning.

NumPy and Pandas

NumPy gives you fast numerical arrays; Pandas builds the DataFrame on top — a labeled table (like a spreadsheet in code) that is the center of gravity of all analysis in Python. You load data into a DataFrame and every later step operates on it.

First five moves on any dataset

These five lines answer: how big is it, what do the columns mean, what types are they, and where are the holes. You never model data you have not first looked at this way.

df.shape         # (rows, columns) — how big is it?
df.head()        # eyeball the first rows
df.info()        # column types + non-null counts
df.describe()    # mean/std/min/quartiles for numbers
df.isna().sum()  # how many missing values per column

Select, filter, group

The three verbs you use constantly: pick columns/rows, keep rows that meet a condition, and summarize by category. The groupby pattern mirrors SQL's GROUP BY.

df[["city", "price"]]                       # select columns
df[df["price"] > 1_000_000]                 # filter rows
df.groupby("city")["price"].mean()          # summarize per city

Missing values and outliers

• Missing data: decide per column — drop the rows, or fill (impute) with the mean/median/mode, or a domain value. Never ignore it silently.
• Outliers: values far from the rest. Sometimes errors to fix, sometimes the most interesting signal (fraud!). Investigate before deleting.
• A boxplot or the IQR rule (1.5 × interquartile range) is the standard way to flag them.

Match the chart to the question

• Comparison across categories → bar chart.
• Trend over time → line chart.
• Relationship between two numbers → scatter plot.
• Distribution of one number → histogram or boxplot.
• Part of a whole → stacked bar (avoid pie charts beyond a few slices).

The tools

Matplotlib is the foundation (full control), Seaborn adds beautiful statistical charts in one line, and Plotly makes interactive charts you can hover and zoom. Streamlit then turns your script into a shareable web app — no web development needed.

import plotly.express as px
fig = px.scatter(df, x="area", y="price", color="city",
                 title="Price vs. area")
fig.show()

Make it honest and clear

A good chart has a clear title, labeled axes, and starts the y-axis at zero for bar charts (truncating it exaggerates differences). Remove clutter; highlight the one thing the reader should see. A visualization is an argument — make it a fair one.

Why models need clean features

Algorithms only understand numbers, and many assume features are on similar scales and free of gaps. Real data is text, categories, missing cells, and wildly different ranges. Preprocessing bridges that gap — 'garbage in, garbage out' is the iron law here.

Encoding categories

Turn text categories into numbers. One-hot encoding makes a 0/1 column per category (best for unordered categories like city). Label/ordinal encoding assigns an order (good for 'small < medium < large').

Scaling numeric features

If 'age' (0–100) and 'income' (0–1,000,000) sit together, distance-based models think income is 10,000× more important just because of its scale. Standardization (subtract mean, divide by std) or min-max scaling (squeeze to 0–1) puts features on equal footing.

Feature engineering: the creative edge

The highest-value step: create new features that expose the signal. From a date, derive day-of-week or is-holiday; from price and area, derive price-per-m². A well-engineered feature can beat a fancier model on raw inputs.

Save the model, not just the notebook

Training and serving are separate. Once trained, you serialize the model to a file (joblib/pickle) so any program can load it and predict without retraining. This artifact is what you deploy.

import joblib
joblib.dump(model, "model.joblib")     # after training
# later, in the app:
model = joblib.load("model.joblib")
model.predict(new_data)

Two ways to serve it

• A web app (Streamlit) — a UI where users type inputs and see predictions. Fastest way to demo value.
• An API (Flask/FastAPI) — an endpoint other software calls to get predictions programmatically. This is how models plug into real products.

It does not end at launch

The world changes, so a live model's accuracy drifts over time (data drift). Production ML means monitoring predictions, retraining on fresh data, and versioning models — the discipline called MLOps, which the advanced program covers in depth.