Autorclone on Colab - Shared Folder to TD Transfer (750GB Limit Bypass) - Video Tutorial

Tutorials & Methods
21

Remote copying, syncing across cloud storages just to mention a few. The best part is that you can do this without utilising your ISP allocated data bandwidth. Allows you to bypass the 750gb daily limit of rclone.
You could refer to the documentation to get more info.

Cheers!!!

… And Over Every Possessor of Knowledge, There is (Some) One (Else) More Knowledgeable.

Ravi_Khatri 22

23

You guys should note that as potential prospective data hoarders, the best is to have multiple backup copies of your files in several cloud storages. This is a fail safe mechanism in case some get taken down. You can achieve this seamlessly using these tools–rclone, Folderclone, Autorclone, gclone or anyone you are comfortable with. It doesn’t cost you any bandwidth, and it is very fast.

It is the best imo. Using mixexplorer or other similar GUI apps have their cons

Cheers!!!

… And Over Every Possessor of Knowledge, There is (Some) One (Else) More Knowledgeable.

24

Thank u so much bro…
but i am little confused here;
what if we have long path?? for example;
Team Drive/uploads/movies/new movies/avangers/

25

I managed to make it work, but i got this error

2020/06/16 01:14:01 Failed to copy with 179 errors: last error was: failed to open source object: open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded

26

Thanks - the colab notebook works.
What would I need to change so that I can copy to a crypt remote so that the files are encrypted?
I already have another rclone config with all my crypted remotes - how can I use that with this colab notebook?

Any help appreciated :mask:

27

There is upload config option i think

28

Hey @FlashyFlash & @Waga
Most Respectfully, I’m getting this error:

rclone is detected: /usr/bin/rclone
generating rclone config file.
rclone config file generated.
Start: 07:22:50
rclone --config ./rclone.conf copy --drive-server-side-across-configs --rc --rc-addr="localhost:5572" -vv --ignore-existing --tpslimit 3 --transfers 3 --drive-chunk-size 32M --drive-acknowledge-abuse --log-file=log_rclone.txt "src001:" "dst001:" &
>> Let us go dst001: 07:22:50
2020/12/18 07:23:00 Failed to rc: connection failed: Post "http://localhost:5572/core/pid": dial tcp 127.0.0.1:5572: connect: connection refused
2020/12/18 07:23:00 Failed to rc: connection failed: Post "http://localhost:5572/core/stats": dial tcp 127.0.0.1:5572: connect: connection refused
2020/12/18 07:23:00 Failed to rc: connection failed: Post "http://localhost:5572/core/stats": dial tcp 127.0.0.1:5572: connect: connection refused
2020/12/18 07:23:00 Failed to rc: connection failed: Post "http://localhost:5572/core/stats": dial tcp 127.0.0.1:5572: connect: connection refused
No rclone task detected (possibly done for this account). (1/3)
rclone --config ./rclone.conf copy --drive-server-side-across-configs --rc --rc-addr="localhost:5572" -vv --ignore-existing --tpslimit 3 --transfers 3 --drive-chunk-size 32M --drive-acknowledge-abuse --log-file=log_rclone.txt "src002:" "dst002:" &
>> Let us go dst002: 07:23:00
2020/12/18 07:23:10 Failed to rc: connection failed: Post "http://localhost:5572/core/pid": dial tcp 127.0.0.1:5572: connect: connection refused
2020/12/18 07:23:11 Failed to rc: connection failed: Post "http://localhost:5572/core/stats": dial tcp 127.0.0.1:5572: connect: connection refused
2020/12/18 07:23:11 Failed to rc: connection failed: Post "http://localhost:5572/core/stats": dial tcp 127.0.0.1:5572: connect: connection refused
2020/12/18 07:23:11 Failed to rc: connection failed: Post "http://localhost:5572/core/stats": dial tcp 127.0.0.1:5572: connect: connection refused
No rclone task detected (possibly done for this account). (2/3)
rclone --config ./rclone.conf copy --drive-server-side-across-configs --rc --rc-addr="localhost:5572" -vv --ignore-existing --tpslimit 3 --transfers 3 --drive-chunk-size 32M --drive-acknowledge-abuse --log-file=log_rclone.txt "src003:" "dst003:" &
>> Let us go dst003: 07:23:11
2020/12/18 07:23:21 Failed to rc: connection failed: Post "http://localhost:5572/core/pid": dial tcp 127.0.0.1:5572: connect: connection refused
2020/12/18 07:23:21 Failed to rc: connection failed: Post "http://localhost:5572/core/stats": dial tcp 127.0.0.1:5572: connect: connection refused
2020/12/18 07:23:21 Failed to rc: connection failed: Post "http://localhost:5572/core/stats": dial tcp 127.0.0.1:5572: connect: connection refused
2020/12/18 07:23:21 Failed to rc: connection failed: Post "http://localhost:5572/core/stats": dial tcp 127.0.0.1:5572: connect: connection refused
No rclone task detected (possibly done for this account). (3/3)
All done (3/3).
Elapsed Time: 00:00:30.40

@FlashyFlash @Waga
I’ve followed the Tutorial Carefully and enabled the apis.

Everything worked till here:

Screenshot (21)
Screenshot (21)958×310 17.8 KB

29

After some searching I found that the rclone commands are in rclone_sa_magic.py file but I’m not sure what to change or add to encrypt the files - also not sure where to add my crypt remote config…

I asked on their telegram group but no luck so far…will update if any progress

30

Looks like a problem with ur config,first check if ur config is working

31

Are you trying to copy from a 3rd party TD as source? If yes, that’s the problem. You need to add your sa to the source TD also.

Cheers!!!

… And Over Every Possessor of Knowledge, There is (Some) One (Else) More Knowledgeable.

32

@Waga Yeah I understood that just after posting the comment, when i read a line saying same as you said, in All_In_One v1.9.8.2 (ipynb).

So, Now my question is : I followed your tutorial in https://edutechtainment.com/ TD to TD using airexplorer, Np++ and rclone. But is there anyway to bypass 750 Gb limit just their in rclone.

33

For rclone, unfortunately there is no way to bypass this limit. Since you followed the tutorial on my blog, you can make use of the script to automate the process. Just run the script daily until you completely copy the content you wish to copy. In addition, you can use multiple email ID on the 3rd party source TD, while maintaining thesame destination TD directory. Rclone will skip the already copied files in thesame destination.

Cheers!!!

… And Over Every Possessor of Knowledge, There is (Some) One (Else) More Knowledgeable.

34

Thanks @Waga, It means a lot when a very busy person reply for just solving someone’s problem (That is unknown to him) جَزَاكَ ٱللَّٰهُ :innocent: :heart_eyes:

35

Exactly :innocent: