This post is provided to help you and for personal use only. Sharing the content of your subscribed materials and the other purchased content is strictly prohibited under 1Hack Terms of Use.
By using this provided material the website 1Hack.us is not responsible for any law infringement caused by the users of this material.
22 April 2020 (v2.3.2)
---------------------
+ added summary
+ added Exception when file is unable to download
21 Aprial 2020 (v2.3.1)
---------------------
While crawling, fetching might cause errors sometime due to some quick requests or server is busy.
This problem has caused the eror in getting a json, so we re-fetch the url again (up to MAX_RETRY_CRAWLING)
or until we found key "files" in the return response. Once retries is reached the maximum and
the key "files" is not found, so we ignore this link (return [])
At the end, if you find there is failure, just re-run the download section again. Unless you set
OVERWITE = TRUE, all files will be re-downloaded
+ added MAX_RETRY_CRAWLING (v2.3)
+ fixed FILE_EXISTING_CHECK (stupid) bug
+ added failure-links download task
20 Aprial 2020 (v2.2)
---------------------
Some sub-folders may be password-protected which will cause the error while crawling, so we skip this folder
+ added auto-skip password-protected folder
17 April 2020 (v2.1)
---------------------
+ fixed URL duplicated when crawling
+ added search 'files' key for some websites do not have proper files structure. So, we search it\
16 April 2020 (v2.0)
---------------------
+ crawler_v2:
* API-based GoIndex crawler
* Collecting all urls to be downloaded
+ parallel downloader
* TDQM progress bar
look like your laughting about my zero level The problem is that i m not english native.
I don’t know very well english. So please tell me the goindex site are where ?
by googling this is a gh link https://index.gd.workers.dev/ but they ask for password
My apology if you think I was laughing, actually I am not laughing. It is my way to teach you guys. Every time I replied, I always convince you guys to
do some research yourself at the beginning before asking or bothering the others.
guide or teach, not to give solutions. (these 2 have completely definitions)
Give a man a fish, and you feed him for a day. Teach a man to fish, and you feed him for a lifetime.
interesting! so where are they?
If you look carefully in my script, you would find out immediately what is the GoIndex site. FYI, once I joined this community I had no idea as well what is the GoIndex website.
Not as always. Some GoIndex website (hint free courses or in my script) are not password-protected .
Note that if you want to download files from password-protected folders, you need to use different solution (probably version 1 + amendments + password from localstorage).
Thank you dear.
I was in a website called coursehunter and after a moment they change all their rules. there were no more free courses after a month. and you have to pay for subscription.
Now I see lot of their course in google shared drive on 1h by some members …by googling i found a site webpremium and when i try to download it s impossible and just stream on the url like lol.coronavirus.worker.dev/coursestitle or worker.dev/coursetitle.
When I compare those addresses with the one in your script i make conclusion that goindex is in relation with free access to paied services cause of the covid-19 pandemic
MAX_DOWNLOAD_TASKS = 32
pool = multiprocessing.Pool(processes=MAX_DOWNLOAD_TASKS) # Num of CPUs
Hi brother, I have a doubt The google
colab free version has only 2 cores by default but in this code it is set 32, could you explain why you chose 32 for the MAX_DOWNLOAD_TASKS ?
The problem is that i m not english native.
!