Archiveteam
/
grab-site
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Commit Graph

  • b297c9e280 README: tweak BrowserStack message master Ivan Kozik 2022-09-01 05:28:58 +0000
  • eed00f234a Revert "README: remove BrowserStack mention" Ivan Kozik 2022-09-01 05:27:57 +0000
  • b659255c1c README: remove old tmux 2.1 note Ivan Kozik 2022-08-08 19:23:47 +0000
  • 32b127067e README: add nix-env install steps for installing the latest version of grab-site Ivan Kozik 2022-08-07 08:49:12 +0000
  • a2e49a1d96 2.2.7 Ivan Kozik 2022-08-07 08:16:46 +0000
  • 077d61af03 Update Firefox UA Ivan Kozik 2022-08-07 08:16:07 +0000
  • 951522ce65 2.2.6 Ivan Kozik 2022-08-07 08:11:46 +0000
  • f04e5f6166 README: update link to ludios_wpull Ivan Kozik 2022-08-07 08:11:09 +0000
  • 5ee2132472 README: document --no-global-igset Ivan Kozik 2022-08-07 08:10:58 +0000
  • d2bf2844dc Fix: when there are no ignores, ignore nothing rather than everything Ivan Kozik 2022-08-07 08:07:59 +0000
  • a178d76ea7 Don't crash when igsets file is empty Ivan Kozik 2022-08-07 07:51:46 +0000
  • 4994331eea Add --no-global-igset for starting a crawl without the "global" ignore set Ivan Kozik 2022-08-07 07:46:55 +0000
  • 0e67d79ae7 2.2.5 Ivan Kozik 2022-07-28 16:15:06 +0000
  • 09b26c88fd global igset: don't ignore Wikipedia thumbnails Ivan Kozik 2022-07-28 16:12:26 +0000
  • a8538b0118 2.2.4 Ivan Kozik 2022-06-27 23:24:30 +0000
  • df06e14415
         Merge pull request #222 from JustAnotherArchivist/warc-header-gs-version Ivan Kozik 2022-06-27 16:15:01 -0700
  • cb477e68a5 README: nixpkgs 21.11 -> 22.05 Ivan Kozik 2022-06-27 23:13:49 +0000
  • a2ca380534 Record grab-site version in WARC headers JustAnotherArchivist 2022-05-07 17:29:51 +0000
  • 9e6e95b513 2.2.3 Ivan Kozik 2022-05-04 20:13:54 +0000
  • 4f2526dbc6 gs-server: fix RuntimeError: To use txaio, you must first select a framework with .use_twisted() or .use_asyncio() Ivan Kozik 2022-05-04 20:12:38 +0000
  • 24a67521ef README: Python 3.8.12 -> 3.8.13 Ivan Kozik 2022-05-04 20:12:31 +0000
  • 53899380a9 README: Debian install: add packages `wget ca-certificates` for the subsequent steps Ivan Kozik 2022-05-04 20:12:04 +0000
  • 9987b25af9 README: add a note Ivan Kozik 2022-05-03 07:23:28 +0000
  • 87c725a333 README: fix anchor link Ivan Kozik 2022-05-03 07:18:26 +0000
  • 20e5fef01d README: remove the Nix-based macOS install because it fails due to Yapsy test failures Ivan Kozik 2022-04-10 18:23:30 +0000
  • 3d2699fb2f README: macOS Nix-based install: no need to edit shell startup files yourself now Ivan Kozik 2022-04-10 17:58:13 +0000
  • 81995d67c2 README: update the macOS Nix-based install Ivan Kozik 2022-04-10 17:57:14 +0000
  • fe006e4fe1 README: fix macOS homebrew-based install: update for Python 3.8 and M1 Macs Ivan Kozik 2022-04-10 17:52:02 +0000
  • 14c3bbdf71 README: nixpkgs 21.05 -> 21.11 Ivan Kozik 2022-01-02 07:21:18 +0000
  • 5a306b6917 README: Python 3.7.11 -> 3.8.12; mention Debian 11 Ivan Kozik 2022-01-02 07:20:34 +0000
  • 41a77f0812 README: have consistency among the 'and then restart your shell' text Ivan Kozik 2022-01-02 07:00:21 +0000
  • d696604509 README: tabs before spaces Ivan Kozik 2022-01-02 06:58:29 +0000
  • 865aae75c9 Update README.md Preservation-Quest 2022-01-01 22:55:21 -0800
  • 9a3e577295 Update README.md Preservation-Quest 2022-01-01 22:51:56 -0800
  • 9f6e012c83 Update README.md Preservation-Quest 2021-12-30 15:19:43 -0800
  • 6269289a2c README: document --wpull-args=--no-warc-compression Ivan Kozik 2021-10-25 19:58:55 +0000
  • 6f4d435bf2
         Merge pull request #203 from TheTechRobo/simplemachineforums-igsets Ivan Kozik 2021-10-25 12:53:33 -0700
  • f0df373701
         Remove printpage from forum igset TheTechRobo 2021-10-25 11:07:34 -0400
  • a376f67130
         Add SimpleMachineForum ignores to `forums` igset TheTechRobo 2021-10-24 18:12:04 -0400
  • 4fbf6469a1 README: fix the nix-based install steps to use release-21.05 because master has the incompatible sqlalachemy 1.4 Ivan Kozik 2021-08-29 04:55:59 +0000
  • 4fec250a32 README: mention Ubuntu 20.04 Ivan Kozik 2021-08-29 04:33:48 +0000
  • bf7da79ce5 2.2.2 Ivan Kozik 2021-08-29 04:25:29 +0000
  • 0f9585db6f README: Python 3.7.10 -> 3.7.11 Ivan Kozik 2021-08-29 04:25:01 +0000
  • b5962676be Use ludios_wpull 3.0.9 to fix https://github.com/ArchiveTeam/ludios_wpull/issues/16 Ivan Kozik 2021-08-29 04:23:40 +0000
  • fe3cc6ab14 README: Python 3.7.9 -> 3.7.10 Ivan Kozik 2021-05-10 00:59:51 +0000
  • 2e3be5dd29 2.2.1 Ivan Kozik 2021-05-10 00:56:09 +0000
  • cb3068c0ca Use ludios_wpull 3.0.8 to fix https://github.com/ArchiveTeam/grab-site/issues/181 Ivan Kozik 2021-05-10 00:55:45 +0000
  • 132064a24e README: Debian-based install: Python 3.7.8 -> 3.7.9 Ivan Kozik 2021-01-03 02:37:11 +0000
  • 8c5681c726 README: macOS: update maximum version Ivan Kozik 2021-01-03 02:35:33 +0000
  • d25e419fb6 README: remove the note about the macOS-specific lxml bug because I can't repro it Ivan Kozik 2021-01-03 02:31:44 +0000
  • f1e6ae4cd1 README: macOS: mention .zshrc Ivan Kozik 2021-01-03 02:29:04 +0000
  • ce93f62a9a README: update both the Homebrew and Nix install steps on macOS Ivan Kozik 2021-01-03 02:26:01 +0000
  • be38f488af README: link to better WARC playback tools Ivan Kozik 2020-08-07 21:25:56 +0000
  • 12e798b075 2.2.0 Ivan Kozik 2020-07-10 23:57:08 +0000
  • 087e145175 wpull_hooks: compile combined ignore with re if re2 fails Ivan Kozik 2020-07-10 23:55:15 +0000
  • e095e1c5b3 README: Python 3.7.7 -> 3.7.8 Ivan Kozik 2020-07-10 23:47:13 +0000
  • 8343916e3c README: Python 3.7.5 -> 3.7.7 Ivan Kozik 2020-06-20 02:22:26 +0000
  • 8365a9d5f5 2.1.19 Ivan Kozik 2020-06-20 02:21:40 +0000
  • 4032234758 Update Firefox UA Ivan Kozik 2020-06-20 02:21:26 +0000
  • e98006f8d0 2.1.18 Ivan Kozik 2019-11-08 06:30:48 +0000
  • 35f6b6bdae Update Firefox UA Ivan Kozik 2019-11-08 06:28:52 +0000
  • 7dfd0faa37 Mention that --finished-warc-dir takes an absolute path Ivan Kozik 2019-11-08 06:27:43 +0000
  • e22dadb759 README: Python 3.7.3 -> 3.7.5 Ivan Kozik 2019-11-08 01:00:44 +0000
  • d418ba4d9b README: add note about having to run apt-get again Ivan Kozik 2019-11-08 00:56:45 +0000
  • 02343335b0 2.1.17 Ivan Kozik 2019-10-20 19:07:22 +0000
  • e5637ca3b1 dashboard: use wss:// if we're behind a reverse proxy serving with https:// Ivan Kozik 2019-10-20 19:04:52 +0000
  • ecea5ec002 Use `--no-binary lxml` now that newer pip broke `--no-binary` Ivan Kozik 2019-10-20 08:38:52 +0000
  • 5024832257 README: link to macOS bug Ivan Kozik 2019-07-30 01:18:58 +0000
  • 7fb9eecbe2 2.1.16 Ivan Kozik 2019-06-08 01:15:53 +0000
  • 6f48d929de Don't retry initial URLs forever Ivan Kozik 2019-06-08 01:14:03 +0000
  • 5e75c56a7d README: Python 3.7.2 -> 3.7.3 Ivan Kozik 2019-04-28 23:21:34 +0000
  • bfe7cae5f8 2.1.15 Ivan Kozik 2019-04-28 23:20:30 +0000
  • e9ebab6f93 default_cookies.txt: skip the age gate on saidit.net Ivan Kozik 2019-04-28 23:19:59 +0000
  • 985e84abf5 README: fix macOS Homebrew install steps Ivan Kozik 2019-04-19 19:54:29 +0000
  • 33b5a98c7d 2.1.14 Ivan Kozik 2019-03-01 06:19:42 +0000
  • d33a42171d setup.py: https://github.com/ludios/wpull -> https://github.com/ArchiveTeam/ludios_wpull Ivan Kozik 2019-03-01 06:18:11 +0000
  • 86280b6dcd README: ludios/grab-site -> ArchiveTeam/grab-site Ivan Kozik 2019-03-01 06:07:18 +0000
  • e67f24c800 README: Python 3.7.1 -> 3.7.2 Ivan Kozik 2019-03-01 06:06:04 +0000
  • bebeb4c579 2.1.13 Ivan Kozik 2019-02-12 11:43:37 +0000
  • 502eccb3eb
         Merge pull request #147 from Fusl/fix-145 Ivan Kozik 2019-02-12 11:42:39 +0000
  • fe814bf294
         removed references for --process-dependency-links Katie Holly 2019-02-12 12:36:09 +0100
  • bb020e741d
         Reference git repo in install_requires Fixes #145 Katie Holly 2019-02-12 12:00:46 +0100
  • 5cbce99e64 README: tweak indentation Ivan Kozik 2018-12-18 10:22:47 +0000
  • ab8009e076 README: update the upgrade steps Ivan Kozik 2018-12-18 10:21:16 +0000
  • 0dbfdec3ad README: add alternate Nix-based install steps for macOS Ivan Kozik 2018-12-18 10:18:10 +0000
  • 434ee19be0 README: update NixOS & other distribution install instructions Ivan Kozik 2018-12-18 10:13:34 +0000
  • a9727902cd Add back the tumblr Googlebot UA instruction now that it works again; bump Firefox UA Ivan Kozik 2018-12-16 00:19:25 +0000
  • d5b7aa6717 2.1.11 Ivan Kozik 2018-12-14 04:17:08 +0000
  • e7b99e761e Fix tests/online-tests Ivan Kozik 2018-12-14 03:44:53 +0000
  • 2524e9c5e7 Make online tests use localhost only, and factor out tests that require no networking at all Ivan Kozik 2018-12-14 03:40:24 +0000
  • 8d1402e410 2.1.10 Ivan Kozik 2018-12-14 02:17:16 +0000
  • 9d121a80af Tweak .editorconfig Ivan Kozik 2018-12-12 01:20:40 +0000
  • ac2c5d97ee README: Python 3.7.0 -> 3.7.1 Ivan Kozik 2018-12-08 23:29:36 +0000
  • 4471660656 README: use spaces for indentation to avoid tab-completion on Tab paste Ivan Kozik 2018-10-22 15:25:57 +0000
  • e650278421 README: try to get people to stop installing grab-site as root and explain the pip options Ivan Kozik 2018-10-22 15:24:50 +0000
  • d22ea9ad78 README: add install steps for NixOS 18.09 Ivan Kozik 2018-10-18 15:03:42 +0000
  • 700ea18693 Use ludios/wpull 3.0.7 to fix https://github.com/ludios/grab-site/issues/138 Ivan Kozik 2018-10-16 21:24:44 +0000
  • 50a3db6e59 Use ludios/wpull 3.0.6 Ivan Kozik 2018-10-16 16:05:06 +0000
  • 0c18c9c0ea Alignment Ivan Kozik 2018-10-14 10:11:33 +0000
  • 4f898d2b06 README: mention pkg-config Ivan Kozik 2018-10-09 18:32:52 +0000