Commit Graph

23 Commits

Author SHA1 Message Date
Hugo Peixoto 23574c7790 Fix query to work in postgresql 2023-08-01 19:58:31 +01:00
Hugo Peixoto c9ee626258 Docker-ignore git 2023-08-01 17:22:14 +01:00
Hugo Peixoto 8c85a64375 Add postgresql support 2023-08-01 17:20:10 +01:00
Hugo Peixoto d1a4ea3d16 Retry inserts, lazy patch to solve sqlite3 concurrency 2023-08-01 14:21:01 +01:00
Hugo Peixoto 53fc8b74bb Batch large requests into batches of 1000 2023-08-01 02:46:35 +01:00
Hugo Peixoto a55294e7e4 Allow checking for unsubmitted video files 2023-08-01 02:04:25 +01:00
Hugo Peixoto 95c8b982bf Fix download prioritization 2023-08-01 01:14:16 +01:00
Hugo Peixoto aa8a920288 Add bytes and count per contact 2023-07-30 17:45:26 +01:00
Hugo Peixoto fc7eb83800 Fix download instructions 2023-07-30 17:38:35 +01:00
Hugo Peixoto a50101d2bc Add estimated total size to stats section 2023-07-30 01:31:03 +01:00
Hugo Peixoto f8e2945266 Prioritize videos that haven't been downloaded yet 2023-07-30 01:13:02 +01:00
Hugo Peixoto fc9d25ba0d Rework statistics 2023-07-28 20:50:22 +01:00
Hugo Peixoto 2f632491bc Add scraping process details 2023-07-28 20:31:14 +01:00
Hugo Peixoto bbafcfa40f Humanizes bytesizes 2023-07-28 12:00:41 +01:00
Hugo Peixoto 1a81e55723 Add detailed instructions 2023-07-28 11:35:42 +01:00
Hugo Peixoto 5b403f26af Add licensing information 2023-07-28 11:35:31 +01:00
Hugo Peixoto add1def19f Add missing file 2023-07-28 11:11:05 +01:00
Hugo Peixoto 45677f30ba Use puma directly 2023-07-28 11:09:23 +01:00
Hugo Peixoto 91e1f3cf1d Adds readme 2023-07-28 10:52:38 +01:00
Hugo Peixoto b44c11dadf Add more crawler scripts 2023-07-28 10:48:07 +01:00
Hugo Peixoto 5dd3d58072 Add crawler scripts 2023-07-28 10:47:33 +01:00
Hugo Peixoto b5516e7d82 Add dockerfiles 2023-07-28 10:37:52 +01:00
Hugo Peixoto 5a10cead6b Coisas a funcionar 2023-07-28 03:01:40 +01:00