[25.1] Optimize /api/invocations/steps/{step_id}

[mvdbeek]

Bypasses ORM model and gets required attributes directly.

Should be faster and more memory efficient.

How to test the changes?

(Select all options that apply)

• I've included appropriate automated tests.
• This is a refactoring of components with existing test coverage.
• Instructions for manual testing are as follows:
  1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

• I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

[galaxyproject-sentryintegration]

🔍 Existing Issues For Review

Your pull request is modifying functions with the following pre-existing issues:

📄 File: lib/galaxy/model/init.py

Function	Unhandled Issue
to_dict	AttributeError: 'NoneType' object has no attribute 'uuid' /api/jobs... Event Count: 5
to_dict	AttributeError: 'NoneType' object has no attribute 'uuid' /api/dataset_collections/... Event Count: 3
to_dict	AttributeError: 'NoneType' object has no attribute 'uuid' /api/histories/{history_id}/content... Event Count: 1

[natefoo]

This has fixed a huge memory problem on .org, works great.

[github-actions]

This PR was merged without a "kind/" label, please correct.

[ahmedhamidawan]

Locally, running release_25.1, I am seeing this error on several invocations. I was wondering whether it is related to this change?

[SQL: SELECT job.id AS job_id, job.create_time AS job_create_time, job.update_time AS job_update_time, job.history_id AS job_history_id, job.library_folder_id AS job_library_folder_id, job.tool_id AS job_tool_id, job.tool_version AS job_tool_version, job.galaxy_version AS job_galaxy_version, job.dynamic_tool_id AS job_dynamic_tool_id, job.state AS job_state, job.info AS job_info, job.copied_from_job_id AS job_copied_from_job_id, job.command_line AS job_command_line, job.dependencies AS job_dependencies, job.job_messages AS job_job_messages, job.param_filename AS job_param_filename, job.runner_name AS job_runner_name_1, job.job_stdout AS job_job_stdout, job.job_stderr AS job_job_stderr, job.tool_stdout AS job_tool_stdout, job.tool_stderr AS job_tool_stderr, job.exit_code AS job_exit_code, job.traceback AS job_traceback, job.session_id AS job_session_id, job.user_id AS job_user_id, job.job_runner_name AS job_job_runner_name, job.job_runner_external_id AS job_job_runner_external_id, job.destination_id AS job_destination_id, job.destination_params AS job_destination_params, job.object_store_id AS job_object_store_id, job.imported AS job_imported, job.params AS job_params, job.handler AS job_handler, job.preferred_object_store_id AS job_preferred_object_store_id, job.object_store_id_overrides AS job_object_store_id_overrides, job.tool_request_id AS job_tool_request_id
FROM job
WHERE job.id = %(pk_1)s]
[parameters: {'pk_1': 5109}]
(Background on this error at: https://sqlalche.me/e/20/f405)
uvicorn.access INFO 2025-11-11 10:21:30,546 [pN:main.1,p:25928,tN:MainThread] 127.0.0.1:52850 - "GET /history/current_history_json?since=2025-08-05T17:09:46.005909 HTTP/1.1" 200
uvicorn.access INFO 2025-11-11 10:21:31,426 [pN:main.1,p:25928,tN:MainThread] 127.0.0.1:52854 - "GET /api/invocations/fd1db96475b04c67/step_jobs_summary HTTP/1.1" 500
[2025-11-11 10:21:31 -0600] [25928] [ERROR] Exception in ASGI application
Traceback (most recent call last):
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.UndefinedColumn: column job.params does not exist
LINE 1: ...ob_object_store_id, job.imported AS job_imported, job.params...
                                                             ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/fastapi/applications.py", line 1133, in __call__
    await super().__call__(scope, receive, send)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette_context/middleware/raw_middleware.py", line 94, in __call__
    await self.app(scope, receive, send_wrapper)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 182, in __call__
    with recv_stream, send_stream, collapse_excgroups():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/_utils.py", line 85, in collapse_excgroups
    raise exc
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 184, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/home/ahmedhamidawan/repos/galaxy2/lib/galaxy/webapps/galaxy/fast_app.py", line 107, in add_x_frame_options
    response = await call_next(request)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 159, in call_next
    raise app_exc
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 144, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/routing.py", line 716, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/routing.py", line 736, in app
    await route.handle(scope, receive, send)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/routing.py", line 290, in handle
    await self.app(scope, receive, send)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 123, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 109, in app
    response = await f(request)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 387, in app
    raw_response = await run_endpoint_function(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 290, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/starlette/concurrency.py", line 38, in run_in_threadpool
    return await anyio.to_thread.run_sync(func)
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2485, in run_sync_in_worker_thread
    return await future
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 976, in run
    result = context.run(func, *args)
  File "/home/ahmedhamidawan/repos/galaxy2/lib/galaxy/webapps/galaxy/api/workflows.py", line 1715, in invocation_step_jobs_summary
    step_jobs_summary = self.invocations_service.show_invocation_step_jobs_summary(trans, invocation_id)
  File "/home/ahmedhamidawan/repos/galaxy2/lib/galaxy/webapps/galaxy/services/invocations.py", line 194, in show_invocation_step_jobs_summary
    return fetch_job_states(trans.sa_session, ids, types)
  File "/home/ahmedhamidawan/repos/galaxy2/lib/galaxy/managers/jobs.py", line 1517, in fetch_job_states
    job_summaries[job_id] = summarize_jobs_to_dict(sa_session, sa_session.get(Job, job_id))
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/orm/scoping.py", line 1058, in get
    return self._proxied.get(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3694, in get
    return self._get_impl(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3873, in _get_impl
    return db_load_fn(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/orm/loading.py", line 695, in load_on_pk_identity
    session.execute(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2355, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
  File "/home/ahmedhamidawan/repos/galaxy2/.venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column job.params does not exist
LINE 1: ...ob_object_store_id, job.imported AS job_imported, job.params...

[mvdbeek]

No, that looks like a future migration running on old code. Job.params was dropped recently

[martenson]

On 25.1 I see column job.params does not exist even after running sh manage_db.sh downgrade release_25.1 from latest dev. I guess migrations do not recreate the column when downgrading -- is that expected?

[mvdbeek]

Migration is e8e5e0f. if down revision is a merge revision it may not re-add the column. Unrelated to this PR though.