Python virtual environments explained: a practical guide

After 92 days of building Python backends, you've likely hit a problem: installing a package breaks another project, or your local setup differs from production, or a colleague can't run your code because their Python versions don't match. Python virtual environments explained: a practical guide solves this. Virtual environments are the foundation of reproducible, team-friendly Python development—and we'll show you exactly how to use them in production scenarios.

Isolates package versions per project

Your system Python is global. Everything installed there lives in one place, and version conflicts are inevitable. If project A needs Django 3.2 and project B needs Django 4.0, you can't satisfy both without a mechanism to separate them.

Virtual environments create an isolated Python installation per project. Each project gets its own site-packages directory, its own Python executable, and its own dependency tree. This isolation is non-negotiable for team development.

Here's why this matters in practice: imagine you're working on a data pipeline using NumPy 1.20 and scikit-learn 0.24. A colleague starts a new API project and installs NumPy 2.0, which introduces breaking changes. Without virtual environments, both projects are now competing for the same NumPy installation. With virtual environments, each project lives in its own sandbox.

Creating a virtual environment is straightforward:

python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

That's it. You now have a completely isolated Python environment. Any pip install commands will only affect this project. When you activate the environment (the source command), your shell uses the isolated Python executable instead of the system one.

Verify it's working:

which python  # Shows path inside your project directory
python --version
pip list  # Should be mostly empty

The isolation is complete. Install Flask in one environment, and it won't appear in another. This is how we prevent the "works on my machine" problem.

Never install into the system Python

Installing packages into your system Python is a path to chaos. The system Python is owned by your operating system. It may be used by system tools, package managers, or other critical processes. Polluting it with project dependencies creates subtle, hard-to-debug failures.

Consider a production server. If you sudo pip install directly into the system Python and one of those packages has a breaking bug, your entire server becomes unstable. You can't roll back cleanly. You can't test changes safely. You've created a fragile, unreproducible system.

Let's say you're setting up a staging environment. A developer runs:

sudo pip install psycopg2==2.8.6  # Installing globally, to system Python

Six months later, they leave the company. A new team member clones the repository and tries to run the code. They get import errors because nobody documented the system-level dependency. They spend two hours debugging before realizing they need that global package. This is preventable.

The rule is absolute: always use a virtual environment. Make it part of your onboarding process. Include it in your README:

# Setup
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Run
python main.py

Even if you're the only developer on a project, use a virtual environment. It's a habit that scales. When you hand off the code or bring on a team member, they follow the same pattern and immediately have a working environment.

In production, this principle extends further. Docker containers run in isolated environments by default, but we still use virtual environments inside Docker. It's defense in depth—you're explicit about dependencies at every layer.

Commit the lock file, not the venv

Your virtual environment directory (venv/) is build artifacts. Never commit it to version control. It's large, it's platform-specific, and it's reproducible from a lock file.

Instead, commit a lock file: requirements.txt or (better) requirements.lock for pip, or pyproject.toml and poetry.lock for Poetry, or Pipfile.lock for Pipenv.

The lock file records the exact version of every package and its dependencies. It's deterministic. Two developers installing from the same lock file get identical environments, down to patch versions.

Here's a typical workflow with pip and requirements.txt:

# You develop locally, add a package
pip install requests==2.31.0

# Export the exact state
pip freeze > requirements.txt

# Commit requirements.txt
git add requirements.txt
git commit -m "Add requests for HTTP calls"

# Colleague pulls the changes
git pull

# Colleague recreates the environment
pip install -r requirements.txt

# They now have the exact same versions

Add venv/ to your .gitignore:

venv/
*.pyc
__pycache__/
.env
.DS_Store

This approach scales to teams. A lock file is a contract: "if you install this, your environment matches mine." It's reproducibility as infrastructure.

For larger teams or complex projects, consider Poetry or Pipenv. They manage lock files automatically and provide more sophisticated dependency resolution:

# With Poetry
poetry add requests  # Updates pyproject.toml and poetry.lock
git add pyproject.toml poetry.lock
git commit

Poetry's lock file includes transitive dependencies and their exact versions, making it even more deterministic than pip's requirements.txt.

Recreate environments in CI from the lock file

Your CI pipeline should never assume dependencies exist. It should recreate the environment from scratch, just like a new developer onboarding or a production deployment.

This catches a critical class of bugs: when your lock file is outdated or incomplete. If your CI doesn't build environments from the lock file, you won't discover that problem until production.

A simple GitHub Actions workflow:

name: Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      
      # Create fresh virtual environment
      - run: python -m venv venv
      - run: source venv/bin/activate && pip install -r requirements.txt
      
      # Run tests
      - run: source venv/bin/activate && pytest

Notice: we create a fresh virtual environment for every CI run. We install from the lock file. We don't rely on system packages or cached environments (beyond the initial Python setup).

This approach guarantees that your lock file is correct. If you forget to pin a transitive dependency and it breaks, your CI will fail before merging to main.

For production deployments, the same principle applies:

# In your deployment script (Dockerfile, systemd, Lambda, etc.)
python -m venv /opt/app/venv
source /opt/app/venv/bin/activate
pip install -r requirements.lock
gunicorn app:app

Every environment—local, CI, staging, production—is built from the same lock file. This is the only way to achieve reproducibility at scale.

Common mistakes

Forgetting to activate the environment: You create a venv but run python without activating it first. Your packages are installed in the venv, but you're using the system Python. Result: import errors. Always verify with which python—it should show a path inside your project.

Committing venv to git: This bloats your repository and makes it unusable on other platforms. Add venv/ to .gitignore immediately. If you've already committed it, remove it: git rm -r --cached venv && git commit -m "Remove venv".

Using different lock files for dev and production: You have requirements-dev.txt and requirements.txt, but they diverge. CI installs from one, production from another. This creates environment inconsistency. Instead, maintain a single lock file and install optional dev dependencies conditionally if needed.

Not pinning transitive dependencies: You pin Flask==2.0, but Flask depends on Werkzeug, which you don't pin. A new Werkzeug version breaks your code. Use pip freeze to capture the entire tree, or use Poetry, which does this automatically.

Deleting and recreating venv without checking dependencies: A venv can go stale if new dependencies are added to requirements.txt but not installed locally. Before each pull, regenerate: rm -rf venv && python -m venv venv && source venv/bin/activate && pip install -r requirements.txt. Or use a script to make this automatic.

Key takeaways

• Virtual environments isolate project dependencies, preventing version conflicts and making code reproducible across machines.
• Never install packages into system Python. Always activate a project-specific virtual environment first.
• Commit lock files (requirements.txt or poetry.lock) to version control, not the venv directory itself. Lock files are small, readable, and reproducible.
• CI and production environments should build from lock files just like local development, ensuring consistency across all stages.
• Use pip freeze or a tool like Poetry to capture all dependencies, including transitive ones, to prevent surprises in production.
• Make virtual environment setup part of your project's onboarding (document it in the README) so new team members don't skip it.