93b7abccb4
add timeout to requests and optimize directory traversal
...
- Added timeout=(30, 30) to requests.get for improved network robustness
- Avoid unnecessary recursion into excluded directories to reduce API calls
2025-07-21 12:02:40 +01:00
9e82cc8ed3
fix: update file opening encoding to utf-8-sig to handle files with BOM
2025-05-16 10:36:18 +10:00
cce2effd7f
disable cache when retrying
2025-05-05 16:27:23 -04:00
ccd048118a
fix(utils/crawl_github_files): error message
...
The formatted error messages that use undefined variables are fixed and their contents are changed.
2025-04-24 23:26:48 +03:00
03555caa87
refactor(utils/crawl_github_files): deciding default branch
...
Previously the default branch name was considered to be "main", which was wrong.
To fix this, dont put the ref param to query and let Github decide default branch.
2025-04-23 12:21:36 +03:00
ea524efebc
fix(utils/crawl_github_files): branch names that includes "/"
...
To fix this problem, my solution was getting branch names from Github api and checking for matching.
If there is no match, this time check is a tree like this exists.
2025-04-23 12:18:47 +03:00
e7a4931d37
improve error raising
2025-04-22 14:15:23 -03:00
99cc41ea68
update the crawl
2025-04-21 16:03:44 -04:00
ea5e8969f0
support downloading GitHub repo with ssh
2025-04-06 17:10:12 -04:00
dc4d445676
update readme examples
2025-04-04 15:06:41 -04:00
Soua Ali