Skip to content

Add loader status details options to %load_ids#354

Merged
michaelnchin merged 5 commits intomainfrom
load-ids-details
Aug 30, 2022
Merged

Add loader status details options to %load_ids#354
michaelnchin merged 5 commits intomainfrom
load-ids-details

Conversation

@michaelnchin
Copy link
Copy Markdown
Member

Issue #, if available: #350

Description of changes:

  • Added --details option to %load_ids. This displays a table containing status and other details for each bulk load job, as available via the Neptune Load Get-Status API.
  • Added --limit option to %load_ids. When used, the details view will be restricted to only display the statuses for the X most recent bulk load jobs submitted.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@triggan
Copy link
Copy Markdown
Contributor

triggan commented Aug 26, 2022

@michaelnchin - I tested this against a few scenarios and noticed an error when the magic encountered a load with a LOAD_IN_QUEUE status. The error was:

{'error': KeyError('feedCount')}

It appears that the fields that are returned as part of the load status API call are not always included depending on the job status. So feedCount in particular is not returned for a queued job. I don't know what may be different of jobs with other kinds of status.

@michaelnchin
Copy link
Copy Markdown
Member Author

Thanks for the catch, @triggan. I've added a check for the presence of the feedCount object in each load status response.

Copy link
Copy Markdown
Contributor

@triggan triggan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@michaelnchin michaelnchin merged commit 75b7593 into main Aug 30, 2022
@michaelnchin michaelnchin deleted the load-ids-details branch August 30, 2022 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow %load_ids magic to expose load status for each bulk loader job

2 participants