Skip to content

Skip re-upload of unchanged metadata artifacts on creating a publication#7346

Merged
gerrod3 merged 1 commit intopulp:mainfrom
Moustafa-Moustafa:fix/skip-artifact-reupload-on-publish
Mar 12, 2026
Merged

Skip re-upload of unchanged metadata artifacts on creating a publication#7346
gerrod3 merged 1 commit intopulp:mainfrom
Moustafa-Moustafa:fix/skip-artifact-reupload-on-publish

Conversation

@Moustafa-Moustafa
Copy link
Contributor

PublishedMetadata.create_from_file() unconditionally saved artifacts to storage on every publish. When a publication was created for an already-published repository version, this overwrote identical metadata blobs with new timestamps, causing problems with CDN caching layers that use last-modified times.

Applied the same deduplication pattern used for content artifact uploads: look up the artifact by checksum first, verify the file exists in storage, and skip the write if it does.

closes #7344

📜 Checklist

  • Commits are cleanly separated with meaningful messages (simple features and bug fixes should be squashed to one commit)
  • A changelog entry or entries has been added for any significant changes
  • Follows the Pulp policy on AI Usage
  • (For new features) - User documentation and test coverage has been added

See: Pull Request Walkthrough

@Moustafa-Moustafa Moustafa-Moustafa force-pushed the fix/skip-artifact-reupload-on-publish branch from 9a5c3b4 to 123dae0 Compare February 24, 2026 14:37
@gerrod3
Copy link
Contributor

gerrod3 commented Mar 11, 2026

@Moustafa-Moustafa Can you rebase with changes from main? That should fix the CI issues.

PublishedMetadata.create_from_file() unconditionally saved artifacts to storage
on every publish. When a publication was created for an already-published
repository version, this overwrote identical metadata blobs with new timestamps,
causing problems with CDN caching layers that use last-modified times.

Applied the same deduplication pattern used for content artifact uploads: look up
the artifact by checksum first, verify the file exists in storage, and skip the
write if it does.

Assisted-by: GitHub Copilot
closes pulp#7344
@Moustafa-Moustafa Moustafa-Moustafa force-pushed the fix/skip-artifact-reupload-on-publish branch from 123dae0 to 46c4cfc Compare March 11, 2026 19:50
@Moustafa-Moustafa
Copy link
Contributor Author

@Moustafa-Moustafa Can you rebase with changes from main? That should fix the CI issues.

Thanks, Done.

Copy link
Contributor

@gerrod3 gerrod3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@gerrod3 gerrod3 merged commit 862e1c2 into pulp:main Mar 12, 2026
13 checks passed
@patchback
Copy link

patchback bot commented Mar 12, 2026

Backport to 3.63: 💚 backport PR created

✅ Backport PR branch: patchback/backports/3.63/862e1c2209d7a5efb30d3d585b362c4ef5e5b3cc/pr-7346

Backported as #7460

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

@patchback
Copy link

patchback bot commented Mar 12, 2026

Backport to 3.49: 💚 backport PR created

✅ Backport PR branch: patchback/backports/3.49/862e1c2209d7a5efb30d3d585b362c4ef5e5b3cc/pr-7346

Backported as #7462

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

@patchback
Copy link

patchback bot commented Mar 12, 2026

Backport to 3.85: 💚 backport PR created

✅ Backport PR branch: patchback/backports/3.85/862e1c2209d7a5efb30d3d585b362c4ef5e5b3cc/pr-7346

Backported as #7459

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

@patchback
Copy link

patchback bot commented Mar 12, 2026

Backport to 3.105: 💚 backport PR created

✅ Backport PR branch: patchback/backports/3.105/862e1c2209d7a5efb30d3d585b362c4ef5e5b3cc/pr-7346

Backported as #7461

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

@patchback
Copy link

patchback bot commented Mar 12, 2026

Backport to 3.73: 💚 backport PR created

✅ Backport PR branch: patchback/backports/3.73/862e1c2209d7a5efb30d3d585b362c4ef5e5b3cc/pr-7346

Backported as #7463

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

@patchback
Copy link

patchback bot commented Mar 12, 2026

Backport to 3.73: 💔 creation of the backport PR failed

❌ Backport PR branch: patchback/backports/3.73/862e1c2209d7a5efb30d3d585b362c4ef5e5b3cc/pr-7346

Validation Failed: 'A pull request already exists for pulp:patchback/backports/3.73/862e1c2209d7a5efb30d3d585b362c4ef5e5b3cc/pr-7346.'

Backporting merged PR #7346 into main

  1. Ensure you have a local repo clone of your fork. Unless you cloned it
    from the upstream, this would be your origin remote.
  2. Make sure you have an upstream repo added as a remote too. In these
    instructions you'll refer to it by the name upstream. If you don't
    have it, here's how you can add it:
    $ git remote add upstream https://github.com/pulp/pulpcore.git
  3. Ensure you have the latest copy of upstream and prepare a branch
    that will hold the backported code:
    $ git fetch upstream
    $ git checkout -b patchback/backports/3.73/862e1c2209d7a5efb30d3d585b362c4ef5e5b3cc/pr-7346 upstream/3.73
  4. Now, cherry-pick PR Skip re-upload of unchanged metadata artifacts on creating a publication #7346 contents into that branch:
    $ git cherry-pick -x 862e1c2209d7a5efb30d3d585b362c4ef5e5b3cc
    If it'll yell at you with something like fatal: Commit 862e1c2209d7a5efb30d3d585b362c4ef5e5b3cc is a merge but no -m option was given., add -m 1 as follows instead:
    $ git cherry-pick -m1 -x 862e1c2209d7a5efb30d3d585b362c4ef5e5b3cc
  5. At this point, you'll probably encounter some merge conflicts. You must
    resolve them in to preserve the patch from PR Skip re-upload of unchanged metadata artifacts on creating a publication #7346 as close to the
    original as possible.
  6. Push this branch to your fork on GitHub:
    $ git push origin patchback/backports/3.73/862e1c2209d7a5efb30d3d585b362c4ef5e5b3cc/pr-7346
  7. Create a PR, ensure that the CI is green. If it's not — update it so that
    the tests and any other checks pass. This is it!
    Now relax and wait for the maintainers to process your pull request
    when they have some cycles to do reviews. Don't worry — they'll tell you if
    any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Overwriting unchanged metadata artifacts in storage cause CDN caching issues

2 participants