Dependency installation issue in 5.0.0rc2

[iercan]

Bug description

I'm trying testing 5.0.0rc2 while currently using version 4.1.2. We utilize the official Docker image and install dependencies as the superset user in the Dockerfile as shown below.

USER root
RUN apt update && apt install -y python3-dev default-libmysqlclient-dev build-essential pkg-config
USER superset
RUN pip install --no-cache-dir mysqlclient

My Docker image works with version 4.1.2, but running a DB upgrade on 5.0.0rc2 throws the following error.

2025-04-22 10:55:45,212:ERROR:superset.app:Failed to create app
Traceback (most recent call last):
  File "/app/.venv/lib/python3.10/site-packages/superset/app.py", line 40, in create_app
    app_initializer.init_app()
  File "/app/.venv/lib/python3.10/site-packages/superset/initialization/__init__.py", line 465, in init_app
    self.setup_db()
  File "/app/.venv/lib/python3.10/site-packages/superset/initialization/__init__.py", line 661, in setup_db
    pessimistic_connection_handling(db.engine)
  File "/app/.venv/lib/python3.10/site-packages/flask_sqlalchemy/__init__.py", line 998, in engine
    return self.get_engine()
  File "/app/.venv/lib/python3.10/site-packages/flask_sqlalchemy/__init__.py", line 1017, in get_engine
    return connector.get_engine()
  File "/app/.venv/lib/python3.10/site-packages/flask_sqlalchemy/__init__.py", line 594, in get_engine
    self._engine = rv = self._sa.create_engine(sa_url, options)
  File "/app/.venv/lib/python3.10/site-packages/flask_sqlalchemy/__init__.py", line 1027, in create_engine
    return sqlalchemy.create_engine(sa_url, **engine_opts)
  File "<string>", line 2, in create_engine
  File "/app/.venv/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py", line 375, in warned
    return fn(*args, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 544, in create_engine
    dbapi = dialect_cls.dbapi(**dbapi_args)
  File "/app/.venv/lib/python3.10/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 150, in dbapi
    return __import__("MySQLdb")
ModuleNotFoundError: No module named 'MySQLdb'
Traceback (most recent call last):
  File "/app/.venv/bin/superset", line 10, in <module>
    sys.exit(superset())
  File "/app/.venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/app/.venv/lib/python3.10/site-packages/click/core.py", line 1685, in invoke
    super().invoke(ctx)
  File "/app/.venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/app/.venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/flask/cli.py", line 355, in decorator
    app = __ctx.ensure_object(ScriptInfo).load_app()
  File "/app/.venv/lib/python3.10/site-packages/flask/cli.py", line 309, in load_app
    app = locate_app(import_name, name)
  File "/app/.venv/lib/python3.10/site-packages/flask/cli.py", line 238, in locate_app
    return find_app_by_string(module, app_name)
  File "/app/.venv/lib/python3.10/site-packages/flask/cli.py", line 166, in find_app_by_string
    app = attr(*args, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/superset/app.py", line 40, in create_app
    app_initializer.init_app()
  File "/app/.venv/lib/python3.10/site-packages/superset/initialization/__init__.py", line 465, in init_app
    self.setup_db()
  File "/app/.venv/lib/python3.10/site-packages/superset/initialization/__init__.py", line 661, in setup_db
    pessimistic_connection_handling(db.engine)
  File "/app/.venv/lib/python3.10/site-packages/flask_sqlalchemy/__init__.py", line 998, in engine
    return self.get_engine()
  File "/app/.venv/lib/python3.10/site-packages/flask_sqlalchemy/__init__.py", line 1017, in get_engine
    return connector.get_engine()
  File "/app/.venv/lib/python3.10/site-packages/flask_sqlalchemy/__init__.py", line 594, in get_engine
    self._engine = rv = self._sa.create_engine(sa_url, options)
  File "/app/.venv/lib/python3.10/site-packages/flask_sqlalchemy/__init__.py", line 1027, in create_engine
    return sqlalchemy.create_engine(sa_url, **engine_opts)
  File "<string>", line 2, in create_engine
  File "/app/.venv/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py", line 375, in warned
    return fn(*args, **kwargs)
  File "/app/.venv/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 544, in create_engine
    dbapi = dialect_cls.dbapi(**dbapi_args)
  File "/app/.venv/lib/python3.10/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 150, in dbapi
    return __import__("MySQLdb")
ModuleNotFoundError: No module named 'MySQLdb'

Although mysqlclient seem installed with pip freeze, the superset user cannot access it.

I noticed the repository now uses uv. When I tried installing it via uv, a permission error occurred. Installing it as the root user resolved the issue.

@mistercrunch, I was advised to tag you since you migrated the image to uv.

Screenshots/recordings

No response

Superset version

master / latest-dev

Python version

3.10

Node version

16

Browser

Chrome

Additional context

No response

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dosubot]

To resolve the "No module named 'MySQLdb'" error when upgrading to Superset version 5.0.0rc2 using the official Docker image, you can follow these steps:

1. Create a requirements-local.txt file in the Docker directory:

   touch ./docker/requirements-local.txt

2. Add mysqlclient to the file:

   echo "mysqlclient" >> ./docker/requirements-local.txt

3. Launch Superset with the appropriate Docker Compose file:

   • If you are using a stock (non-customized) Superset image, run:

     docker compose -f docker-compose-non-dev.yml up

4. For a customized Docker image, create a custom Dockerfile that includes the necessary build dependencies before installing the mysqlclient package. This is important because starting from Superset version 4.1.0, database drivers were removed from the base Docker image to create a leaner image.

5. Rebuild your local image with the new driver and dependencies included:

   docker compose build --force-rm

   After rebuilding, relaunch Superset by running:

   docker compose up

6. Verify the installation by entering the running container:

   docker exec -it <container_name> bash

   Then run pip freeze to check that the mysqlclient package is listed.

Additionally, the use of uv for package installation in the Superset Docker image can cause permission issues. If you encounter a 'uv: command not found' error, you can modify the installation script to check for the presence of 'uv' and use 'pip' if 'uv' is not available. Here's a snippet to handle this:

if command -v uv > /dev/null 2>&1; then
    uv pip install --no-cache-dir -r "${REQUIREMENTS_LOCAL}"
else
    pip install --no-cache-dir -r "${REQUIREMENTS_LOCAL}"
fi

These steps should help ensure that the mysqlclient package is installed and accessible by the Superset user within the Docker environment ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[rusackas]

Relates to #33225
...not sure at first glance if it's a duplicate though.

[mistercrunch]

Use uv pip install. We introduce this package manager that's much faster than pip. See how it's done in the main Dockerfile

[iercan]

@mistercrunch As I mentioned, If I try to use uv as superset user It throws permission error. I'm able to use it only as root user. I believe this is not the expected behaviour.