X-Git-Url: http://git.hcoop.net/clinton/thingy_grabber.git/blobdiff_plain/fb28c59b2ab135f0e36cc330644543bf54606862..10f0238d9070c7f6443a5b4f3eb6da3d6eba61aa:/README.md diff --git a/README.md b/README.md index a17764a..07520c2 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,10 @@ # thingy_grabber -Script for archiving thingiverse things. Due to this being a glorified webscraper, it's going to be very fragile. +Script for archiving thingiverse things. ## Usage: ```` -usage: thingy_grabber.py [-h] [-l {debug,info,warning}] [-d DIRECTORY] [-f LOG_FILE] [-q] {collection,thing,user,batch,version} ... +usage: thingy_grabber.py [-h] [-l {debug,info,warning}] [-d DIRECTORY] [-f LOG_FILE] [-q] [-c] [-a API_KEY] + {collection,thing,user,batch,version} ... positional arguments: {collection,thing,user,batch,version} @@ -23,8 +24,23 @@ optional arguments: -f LOG_FILE, --log-file LOG_FILE Place to log debug information to -q, --quick Assume date ordering on posts + -c, --compress Compress files + -a API_KEY, --api-key API_KEY + API key for thingiverse ```` +## API KEYs +Thingy_grabber v0.10.0 accesses thingiverse in a _substantially_ different way to before. The plus side is it should be more reliable, possibly faster and no longer needs selenium or a firefox instance (and so drastically reduces memory overhead). The downside is you are _going_ to have to do something to continue using the app - basically get yourself an API KEY. + +To do this, go to https://www.thingiverse.com/apps/create and create your own selecting Desktop app. +Once you have your key, either specify it on the command line or put it in a text file called `api.key` whereever you are running the script from - the script will auto load it. + +### Why can't I use yours? +Because API keys can (are?) rate limited. + +## Downloads +The latest version can be downloaded from here: https://github.com/cwoac/thingy_grabber/releases/. Under the 'assets' triangle there is precompiled binaries for windows (no python needed!). + ### Things `thingy_grabber.py thing thingid1 thingid2 ...` This will create a directory named after the title of the thing(s) with the given ID(s) and download the files into it. @@ -77,7 +93,7 @@ Download all designs by jim and bob into directories under `c:\downloads`, give ` ## Requirements -python3, beautifulsoup4, requests, lxml +python3, requests, py7xr (>=0.8.2) ## Current features: - can download an entire collection, creating seperate subdirs for each thing in the collection @@ -85,6 +101,33 @@ python3, beautifulsoup4, requests, lxml - If there is an updated file, the old directory will be moved to `name_timestamp` where `timestamp` is the last upload time of the old files. The code will then copy unchanged files across and download any new ones. ## Changelog +* v0.10.2 + - Fixed regression in rest API +* v0.10.1 + - A couple of minor bug fixes on exception handling. +* v0.10.0 + - API access! new -a option to provide an API key for more stable access. +* v0.9.0 + - Compression! New -c option will use 7z to create an archival copy of the file once downloaded. + Note that although it will use the presence of 7z files to determine if a file has been updated, it currently _won't_ read old files from inside the 7z for handling updates, resulting in marginally larger bandwidth usage when dealing with partially updated things. This will be fixed later. + - Internal tidying of how old directories are handled - I've tested this fairly heavily, but do let me know if there are issues. +* v0.8.7 + - Always, Always generate a valid time stamp. +* v0.8.6 + - Handle thingiverse returning no files for a thing gracefully. +* v0.8.5 + - Strip '.'s from the end of filenames + - If you fail a download for an already failed download it no longer throws an exception + - Truncates paths that are too long for windows +* v0.8.4 + - Just use unicode filenames - puts the unicode characters back in! + - Force selenium to shutdown firefox on assert and normal exit +* v0.8.3 + - Strip unicode characters from license text +* v0.8.2 + - Strip unicode characters from filenames +* v0.8.1 + - Fix bug on when all files were created / updated in October after the 9th. * v0.8.0 - Updated to support new thingiverse front end * v0.7.0