Certified Data Scientist Expert (Advanced)

Classes and objects

A class is a blueprint; an object is a thing built from it. The class bundles data (attributes) with the functions that act on that data (methods). Instead of loose variables and functions, related state and behavior live together.

class Account:
    def __init__(self, owner, balance=0):
        self.owner = owner          # attribute
        self.balance = balance

    def deposit(self, amount):      # method
        self.balance += amount
        return self.balance

acc = Account("Sara", 100)
acc.deposit(50)                     # 150

The four pillars

• Encapsulation — bundle data with its methods and hide internals.
• Inheritance — a class can extend another, reusing and specializing it.
• Polymorphism — different classes respond to the same method name in their own way.
• Abstraction — expose a simple interface, hide the complex implementation.

Why it matters for data science

scikit-learn, PyTorch and Pandas are all built with OOP — every model is an object with .fit() and .predict(). Understanding classes lets you build custom transformers, extend library classes, and structure a project so it stays maintainable as it scales.

How a scrape works

You request a page's HTML, then parse it to pull out the parts you want. requests fetches the HTML; BeautifulSoup navigates the tag tree to extract text and links by their tags, classes or ids.

import requests
from bs4 import BeautifulSoup

html = requests.get("https://example.com/jobs").text
soup = BeautifulSoup(html, "html.parser")
titles = [h.text.strip() for h in soup.select("h2.job-title")]

Static vs. dynamic pages

If the data is in the initial HTML, requests + BeautifulSoup is enough. If the page builds content with JavaScript after loading (infinite scroll, dashboards), you need a browser automation tool like Selenium or Playwright that runs the page like a real browser first.

Window functions

A window function computes across a set of rows related to the current row — without collapsing them like GROUP BY does. This is how you get running totals, rankings, and row-to-row comparisons while keeping every row visible.

SELECT name, city, total,
       RANK() OVER (PARTITION BY city ORDER BY total DESC) AS city_rank,
       SUM(total) OVER (PARTITION BY city) AS city_total
FROM   customers;

CTEs make complex queries readable

A Common Table Expression (WITH …) names a subquery so you can build a query in clear, stackable steps instead of deeply nested parentheses. Complex analytics become a readable pipeline — and CTEs can even be recursive for hierarchies (org charts, categories).

WITH monthly AS (
  SELECT DATE_FORMAT(created, '%Y-%m') AS m, SUM(total) AS rev
  FROM orders GROUP BY m
)
SELECT m, rev, rev - LAG(rev) OVER (ORDER BY m) AS growth
FROM monthly;

Stored logic & performance

• Stored procedures & functions save reusable logic inside the database.
• Triggers run automatically on INSERT/UPDATE/DELETE (audit logs, validation).
• Indexes make lookups fast; read a query's EXPLAIN plan to find slow spots.

The problem: memory & one core

Pandas loads the whole dataset into RAM and uses one CPU core. A 50 GB file on a 16 GB laptop simply won't open, and big operations are slow. Big-data tools solve both by splitting the work.

How Dask helps

A Dask DataFrame is many Pandas DataFrames under one interface. It splits data into partitions and processes them in parallel, only loading what it needs. The API mirrors Pandas, so your existing skills transfer almost unchanged.

import dask.dataframe as dd
df = dd.read_csv("huge/*.csv")          # lazy — nothing loaded yet
result = df.groupby("city").amount.mean()
result.compute()                        # now it runs, in parallel

Imbalanced classes: SMOTE

When one class is rare (1% fraud), a model can score high by ignoring it. SMOTE and ADASYN fix this by synthesizing new, plausible minority examples between existing ones — balancing the classes so the model actually learns the rare pattern. Resample only the training set, never the test set.

Turning text into numbers

Models need numbers, so text must be vectorized. TF-IDF weights words by how distinctive they are to a document. Word2Vec goes further — it learns dense embeddings where words with similar meaning sit close together, capturing semantics a bag-of-words misses.

Time-series features

• Lag features — yesterday's value as a predictor of today's.
• Rolling windows — moving averages/std smooth noise and expose trend.
• Calendar features — day-of-week, month, holiday flags capture seasonality.

Ensembles: wisdom of crowds

One model can be biased or noisy; many diverse models averaged together are usually more accurate and stable. Bagging (e.g. Random Forest) trains models in parallel on random data subsets and averages them. Boosting trains models in sequence, each one fixing the previous one's mistakes.

XGBoost — the competition favorite

Gradient boosting — and its fast implementation XGBoost (with LightGBM and CatBoost) — dominates tabular-data problems. It builds shallow decision trees one after another, each correcting the residual errors of the ensemble so far. It's often the strongest model you can reach for on structured data.

PCA & association rules

• PCA reduces many correlated features to a few 'principal components' that keep most of the variance — great for visualization and speed.
• DBSCAN clusters by density, finding arbitrary shapes and flagging outliers (unlike k-means).
• Association rules (Apriori / FP-Growth) find 'customers who buy X also buy Y' — market-basket analysis.

Docker: 'works on my machine' — solved

A container packages your code with its exact dependencies and environment, so it runs identically on your laptop, a colleague's machine, and a server. No more version mismatches. You write a Dockerfile once; anyone can build and run the same image.

# Dockerfile
FROM python:3.11-slim
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "app.py"]

# build & run
docker build -t my-model .
docker run -p 8000:8000 my-model

Cloud & scaling

Deploy the container to a cloud (AWS, Azure, GCP). For traffic that spikes, an orchestrator like Kubernetes runs many copies and load-balances across them, adding or removing copies automatically. This is the operational backbone of real ML products.