NumPy is great when every value in the array is the same type and the columns don't need names. Real data isn't usually like that. A list of orders has a string customer name, a float total, an integer quantity, a timestamp, and a category code, all in the same row. Pandas is the library that sits on top of NumPy and adds labeled rows, named columns, mixed types per column, and a pile of methods for loading, cleaning, filtering, grouping, and saving tabular data. Anything that starts life as a CSV, an Excel file, or a SQL query usually fits into a pandas DataFrame.

Pandas wraps NumPy arrays in two structures that solve problems NumPy doesn't.

The first is the Series, a 1D labeled array. It's a NumPy array plus an index. The index gives each value a label, which can be an integer, a string, a timestamp, or anything else hashable. Without the index, values are referenced by position; with it, they are referenced by name.

The second is the DataFrame, a 2D labeled table. Each column is a Series, and the columns can have different dtypes. The whole table shares a single row index. A DataFrame is roughly the in-memory equivalent of a SQL table, an Excel sheet, or a CSV file.

The path from raw NumPy to tabular data is two steps. Adding labels to a 1D array produces a Series. Stacking multiple Series side by side with a shared row index produces a DataFrame. Most pandas code uses DataFrames; Series shows up when pulling a single column out of a DataFrame or working with a 1D dataset directly.

Pandas also adds capabilities NumPy doesn't have: graceful missing-value handling (with NaN), per-column dtypes including strings and datetimes, group-by operations (the SQL equivalent), time-series support (date ranges, resampling), and direct readers for every common file format. The pandas conventional import is:

pd is the standard alias. Like with np, every tutorial and Stack Overflow answer uses it, so following the convention keeps code recognizable.

The Series is the simplest pandas object. Create one from a list, a tuple, a NumPy array, or a dict.

The Series printed with two columns: the index on the left (0, 1, 2, 3, 4 by default, just like a list), and the values on the right. The dtype is float64, inferred from the input. Compared to a NumPy array, the index is the addition.

The index becomes interesting when set to something meaningful:

Now each price is labeled by its product name. Looking up a price by name works the same way as a dict lookup:

prices["Mouse"] returns a scalar (the value at that label). prices[["Mouse", "Hub"]] returns a Series with just those two labels, in that order. The square brackets with a list inside is the same fancy-indexing trick from NumPy, now driven by labels instead of integers.

A Series built from a dict picks up the dict's keys as its index automatically:

This is the easiest way to build a labeled Series from existing data.

Arithmetic on a Series works element-wise, like NumPy, and is aware of the index. Adding two Series with the same labels lines them up correctly even if the labels are in different orders:

The output is in sorted-key order, not the input order, and the labels were matched even though they appeared in different positions in each Series. NumPy can't do this because NumPy arrays don't carry labels; positions are all they have. Pandas' label-aware arithmetic is what fits joining real-world data where column order isn't guaranteed.

A DataFrame is a 2D table. Each column has a name and a dtype, and all columns share a single row index. The most common way to build one is from a dict of lists, where each key is a column name and each value is the column's data.

Four columns, five rows, each cell with its proper type. The leftmost column (0 through 4) is the row index, the default RangeIndex that appears when none is specified.

A DataFrame can also be built from other sources:

The DataFrame inferred the columns from the dict keys in the first case, and used the explicit columns argument in the second. Lists of dicts are convenient for ad-hoc construction; the dict-of-lists form is more efficient when whole columns are already in hand.

The row index can be set explicitly with index=... or after the fact with df.set_index("column"). A meaningful index is useful when one column uniquely identifies each row, like a product ID or a timestamp.

product_id is now the index, not a column. Rows can be looked up by ID directly: products.loc["P003"] returns the HDMI Cable row. The original integer position index is gone, but iloc still works for position-based access.

A standout feature of pandas for everyday work is how easy it is to load data from any common source. The reader functions live at the top of the pd namespace.

Every reader returns a DataFrame. The most common is pd.read_csv, which handles the awkward parts of CSV parsing: type inference (price columns come back as floats, count columns as ints), automatic conversion of empty strings to NaN, and optional date parsing.

For a self-contained example without an external file, write a small CSV first and read it back:

read_csv looked at each column and picked a dtype: object (pandas' name for arbitrary Python objects, almost always strings) for the text columns, float64 for prices, int64 for stock counts. No type hints needed; the inference is usually right.

The same data can be written back out with to_csv:

index=False tells pandas not to write the row index to the file. Without it, the CSV gets an extra leading column with the integer positions, which is usually noise that confuses downstream consumers. The same option exists for to_excel, to_json, and the other writers.

The mirror functions for the readers are to_csv, to_excel, to_json, to_sql. Same pattern: call the method on the DataFrame, pass a path or connection.

pd.read_csv is memory-eager: it loads the whole file into RAM and infers dtypes by scanning all rows. For a 10 GB CSV, this is a problem. Use chunksize=N to read in batches, usecols=[...] to load only the needed columns, or dtype={...} to skip the inference step.

The first step with a freshly loaded DataFrame is to look at it. The five methods used every time are head, tail, info, describe, and shape/dtypes.

head(n) shows the first n rows (default 5):

tail(n) shows the last n rows. head and tail together provide a quick sanity check after loading: does the data look sensible at both ends?

shape is a tuple of (rows, columns), just like NumPy's shape:

info() prints a summary: row count, column names, dtypes, non-null counts, memory usage:

The non-null count immediately shows whether any column has missing values (it would show fewer than 50 here). The dtypes confirm whether pandas inferred each column correctly. Memory usage helps reason about whether the table will fit in RAM. This is the second call after head, every time.

describe() gives summary statistics for all numeric columns:

Eight numbers per column: count, mean, std, min, 25th percentile, median (50%), 75th percentile, max. The shape of the distribution in one call. For a freshly loaded dataset, this surfaces outliers, missing data, columns that should be categorical but were inferred as numeric, and dozens of other surprises.

Most data analysis sessions start with these eight calls before anything else. Skipping them turns a two-second df.info() check into an hour of downstream debugging.

Pulling a single column out of a DataFrame uses bracket notation with the column name:

A single column comes back as a Series, with the same row index as the parent DataFrame. The Series knows its name (price), which is useful for plots and labeled output. Multiple columns come back as a DataFrame:

The double brackets are the same fancy-indexing pattern: the inner brackets build a list of column names, the outer brackets do the selection. Single bracket gives one column, double bracket gives a DataFrame with those columns.

For rows, the two preferred indexers are loc (label-based) and iloc (position-based):

loc and iloc give the same result here because the second-position row also happens to be the one labeled P002. The distinction matters when the index is non-sequential, when rows have been filtered out, or when the label looks like an integer (timestamps, custom IDs).

Both indexers accept slices and lists, just like Series:

One thing to watch: loc slicing includes the endpoint, but iloc slicing does not. Labels are arbitrary (there's no "next label" rule the way there is for positions), so excluding a known endpoint would be inconsistent. Once the difference is known, it's hardly ever a problem.

Both indexers also accept a comma to select rows and columns at once:

This is roughly the SQL equivalent of SELECT price, stock FROM products WHERE id IN ('P001', 'P003').

The other common row operation is boolean filtering, which works the same way it does in NumPy: build a boolean Series of the same length as the DataFrame's index, pass it as the row index.

Same rules as NumPy: combine conditions with & (and), | (or), ~ (not), and wrap each comparison in parentheses. The familiar Python and/or won't work because pandas Series with more than one element can't be coerced to a single bool.

For string columns, a whole family of .str.* methods applies per-element:

.str.contains, .str.startswith, .str.lower, .str.replace, and many more behave like the Python string method versions but operate on the whole column. This handles "row matches a text pattern" without writing a loop.

A column is added by assigning to it like a dict key. The values can be a scalar (broadcast across all rows), a list of the right length, a NumPy array, or a Series:

The arithmetic is element-wise: products["price"] * products["stock"] multiplies row by row. The result is a Series with the same index, which pandas drops into the new value column. Almost all "computed columns" are built this way. No loops, just labeled arithmetic.

If a column already exists, the same assignment overwrites it. There's no separate "update" call.

A brief comparison clarifies what pandas is by what it isn't.

Pandas vs SQL. A DataFrame is roughly a SQL table held in memory. The operations map onto SQL fairly directly:

The big difference is that SQL operates on data on disk in the database, and pandas operates on data in the process's memory. For data that fits in RAM, pandas is faster and more expressive. For data that doesn't, SQL (or a tool that talks to a database) fits better.

Pandas vs Excel. Excel is a DataFrame with a UI and formulas. The structural similarity is real: rows, columns, cells, named ranges, sorting, filtering, pivot tables. The difference is that Excel is a spreadsheet with formulas inside cells, while pandas is code that runs on the whole table at once. Excel maxes out around a million rows; pandas handles ten or a hundred million on a laptop. Excel formulas suit visible, manual workflows; pandas code suits repeatable, automated workflows.

Pandas vs raw Python. A list of dicts is a reasonable representation for tabular data when there are 50 rows and the operations are simple. Pandas pays off when there are thousands of rows, the operations are non-trivial (filtering, grouping, joining), or fast aggregations across columns are needed. For one-off "open a JSON file and print three values" scripts, pandas is overkill.

Most everyday data work falls into the pandas bucket. Numbers-only and high-performance lands in NumPy. Datasets too big for memory land in SQL or distributed compute. The boundaries are fuzzy; many real workflows mix two or all three.

To pull the pieces together, consider a small but realistic flow: load orders from a CSV string, inspect them, filter, add a computed column, group by customer, and write the result back out.

Six steps, six lines of pandas. The same flow in raw Python with csv.DictReader and a defaultdict would be three times as long, slower, and more error-prone. The groupby line in particular justifies the pandas dependency: it's the equivalent of SQL's SELECT customer, SUM(total) FROM orders GROUP BY customer ORDER BY SUM(total) DESC, expressed in a form that reads naturally once the grammar is familiar.

This is the rhythm of pandas code. Load with read_csv (or another reader). Inspect with head, info, and describe. Compute new columns. Filter. Group, aggregate, sort. Write back out. The exact set of methods is small, the combinations cover most of data work. Subsequent lessons go deeper on the individual pieces; this is the shape they fit into.