rarias/jungle
2
0
Fork 3
Code Issues 57 Pull Requests 7 Actions Packages Projects Releases Wiki Activity

Add script to trim the repository #188

Manually merged
rarias merged 2 commits from trim-repository into master 2025-10-02 15:46:03 +02:00
Conversation 6 Commits 2 Files Changed 3 +56 -4
rarias commented 2025-10-02 13:09:12 +02:00
Owner
Copy Link

Instead of using a PR that makes it hard to tell what is going on, I would like to instead agree on a script that would trim the repo, and then we can agree if the result is fine. Then we simply push it as the new master branch. It also keeps the record of what we used in case we need to revisit it in the future. The script needs the git-filter-repo command, available in nixpkgs.

The new jungle repository is almost 25 times smaller (57M -> 2,3M) and the content of the current HEAD (which is what nix copies to the store) is 43 times smaller (30M -> 700K).

The new master branch is here: https://jungle.bsc.es/git/rarias/jungle/src/branch/new-master/
The website is available here: https://jungle.bsc.es/git/rarias/jungle-website

The author and commit dates are preserved after the rewrite:

apex% git -C oldjungle log --format=fuller | head
commit 00fe0f46a1143e23d334250dc47fdee2a9320221
Author:     Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
AuthorDate: Tue Sep 30 18:26:33 2025 +0200
Commit:     Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
CommitDate: Wed Oct 1 12:27:43 2025 +0200

    Add acinca user

    Reviewed-by: Aleix Boné <abonerib@bsc.es>

apex% git -C newjungle log --format=fuller | head
commit b040bebd1d0981b8fb6967ef33cb1d4d5eef318b
Author:     Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
AuthorDate: Tue Sep 30 18:26:33 2025 +0200
Commit:     Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
CommitDate: Wed Oct 1 12:27:43 2025 +0200

    Add acinca user

    Reviewed-by: Aleix Boné <abonerib@bsc.es>

Fixes: #171

Instead of using a PR that makes it hard to tell what is going on, I would like to instead agree on a script that would trim the repo, and then we can agree if the result is fine. Then we simply push it as the new master branch. It also keeps the record of what we used in case we need to revisit it in the future. The script needs the `git-filter-repo` command, available in nixpkgs. The new jungle repository is almost 25 times smaller (57M -> 2,3M) and the content of the current HEAD (which is what nix copies to the store) is 43 times smaller (30M -> 700K). The new master branch is here: https://jungle.bsc.es/git/rarias/jungle/src/branch/new-master/ The website is available here: https://jungle.bsc.es/git/rarias/jungle-website The author and commit dates are preserved after the rewrite: ``` apex% git -C oldjungle log --format=fuller | head commit 00fe0f46a1143e23d334250dc47fdee2a9320221 Author: Rodrigo Arias Mallo <rodrigo.arias@bsc.es> AuthorDate: Tue Sep 30 18:26:33 2025 +0200 Commit: Rodrigo Arias Mallo <rodrigo.arias@bsc.es> CommitDate: Wed Oct 1 12:27:43 2025 +0200 Add acinca user Reviewed-by: Aleix Boné <abonerib@bsc.es> apex% git -C newjungle log --format=fuller | head commit b040bebd1d0981b8fb6967ef33cb1d4d5eef318b Author: Rodrigo Arias Mallo <rodrigo.arias@bsc.es> AuthorDate: Tue Sep 30 18:26:33 2025 +0200 Commit: Rodrigo Arias Mallo <rodrigo.arias@bsc.es> CommitDate: Wed Oct 1 12:27:43 2025 +0200 Add acinca user Reviewed-by: Aleix Boné <abonerib@bsc.es> ``` Fixes: #171
rarias added 1 commit 2025-10-02 13:09:13 +02:00
Add script to trim the repository 4a91337da7
rarias added 1 commit 2025-10-02 13:35:42 +02:00
Fetch website from its own git repository e4eab5f027
rarias requested review from abonerib 2025-10-02 13:37:43 +02:00
abonerib reviewed 2025-10-02 13:46:36 +02:00
m/tent/nginx.nix Outdated
@@ -3,3 +3,3 @@
website = pkgs.stdenv.mkDerivation {
name = "jungle-web";
src = theFlake;
src = pkgs.fetchgit {
abonerib commented 2025-10-02 13:45:36 +02:00
Collaborator
Copy Link

We should also fix m/hut/nginx.nix

We should also fix `m/hut/nginx.nix`
rarias commented 2025-10-02 13:49:56 +02:00
Author
Owner
Copy Link

Yes, I think I would just duplicate the changes there for now.

Yes, I think I would just duplicate the changes there for now.
rarias marked this conversation as resolved
rarias force-pushed trim-repository from e4eab5f027 to 4143186bd0 2025-10-02 13:52:35 +02:00 Compare
abonerib reviewed 2025-10-02 15:17:36 +02:00
abonerib left a comment
Collaborator
Copy Link

LGTM, and my results match the sizes reported.

Only issue is that hut and tent configs will be broken from the point where the hugo website was added to nginx till the fix in the latest commit. Do we care about that?

LGTM, and my results match the sizes reported. Only issue is that `hut` and `tent` configs will be broken from the point where the hugo website was added to nginx till the fix in the latest commit. Do we care about that?
rarias commented 2025-10-02 15:27:49 +02:00
Author
Owner
Copy Link

Only issue is that hut and tent configs will be broken from the point where the hugo website was added to nginx till the fix in the latest commit. Do we care about that?

I plan to keep a backup repository in case you need to go back to a previous commit from the old history. I don't think we can keep it in another branch without bloating the clone size. That should cover any of those cases, no?

Here: https://jungle.bsc.es/git/rarias/jungle-backup

> Only issue is that `hut` and `tent` configs will be broken from the point where the hugo website was added to nginx till the fix in the latest commit. Do we care about that? I plan to keep a backup repository in case you need to go back to a previous commit from the old history. I don't think we can keep it in another branch without bloating the clone size. That should cover any of those cases, no? Here: https://jungle.bsc.es/git/rarias/jungle-backup
👍 1
abonerib commented 2025-10-02 15:33:52 +02:00
Collaborator
Copy Link

I plan to keep a backup repository in case you need to go back to a previous commit from the old history. I don't think we can keep it in another branch without bloating the clone size. That should cover any of those cases, no?

Yes, that seems reasonable. The other alternative is to amend the original nginx commits, but it would overcomplicate things.

> I plan to keep a backup repository in case you need to go back to a previous commit from the old history. I don't think we can keep it in another branch without bloating the clone size. That should cover any of those cases, no? Yes, that seems reasonable. The other alternative is to amend the original nginx commits, but it would overcomplicate things.
👍 1
abonerib approved these changes 2025-10-02 15:42:31 +02:00
Dismissed
rarias requested review from abonerib 2025-10-02 15:42:54 +02:00
abonerib approved these changes 2025-10-02 15:43:33 +02:00
rarias force-pushed trim-repository from 4143186bd0 to 5cfd7f0858 2025-10-02 15:45:34 +02:00 Compare
rarias manually merged commit 5cfd7f0858 into master 2025-10-02 15:46:03 +02:00
Sign in to join this conversation.
Reviewers
No Reviewers
abonerib
Labels
Clear labels
bug
ceph
ci
config
cross
hw
io
kernel
mpi
net
nix
ops
repair
slurm
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
abonerib arocanon varcila rarias
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: rarias/jungle#188
Delete Branch "trim-repository"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?