joshuacoles/arc-ingester
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Streamline memory consumption #1

New Issue
Open
opened 2024-04-04 07:40:46 +00:00 by joshuacoles · 0 comments
joshuacoles commented 2024-04-04 07:40:46 +00:00
Owner
Copy Link

src/main.rs Line 96 in 3581797ebe
let bytes = tokio::fs::read(&path).await.unwrap();

Currently we are reading all gzip files before comparing the hashes, this uses needlessly large amounts of memory, hashes can be filtered at the time of computation.

https://git.joshuacoles.me/joshuacoles/arc-ingester/src/commit/3581797ebe63c430a99c07cf68aceb404baa381c/src/main.rs#L96 Currently we are reading all gzip files before comparing the hashes, this uses needlessly large amounts of memory, hashes can be filtered at the time of computation.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: joshuacoles/arc-ingester#1
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?