Xonshiz
/
anime-dl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Command-line program to download anime from CrunchyRoll and Funimation. 

Command-line program to download anime from CrunchyRoll and Funimation.
master
Xonshiz 2017-03-05 00:40:10 +05:30
commit d51e4b8aaf
24 changed files with 8133 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
.gitattributes vendored Normal file
View File

@ -0,0 +1,17 @@
# Auto detect text files and perform LF normalization
* text=auto
# Custom for Visual Studio
*.cs diff=csharp
# Standard to msysgit
*.doc diff=astextplain
*.DOC diff=astextplain
*.docx diff=astextplain
*.DOCX diff=astextplain
*.dot diff=astextplain
*.DOT diff=astextplain
*.pdf diff=astextplain
*.PDF diff=astextplain
*.rtf diff=astextplain
*.RTF diff=astextplain

58
.gitignore vendored Normal file
View File

@ -0,0 +1,58 @@
# Windows image file caches
Thumbs.db
ehthumbs.db
# Folder config file
Desktop.ini
# Recycle Bin used on file shares
$RECYCLE.BIN/
# Windows Installer files
*.cab
*.msi
*.msm
*.msp
*.exe
*.pyc
*.pyz
*.pypirc
*.toc
*.manifest
*.pkg
*.zip
*.xml
*.jpeg
*.jpg
# Windows shortcuts
*.lnk
# =========================
# Operating System Files
# =========================
# OSX
# =========================
.DS_Store
.AppleDouble
.LSOverride
# Thumbnails
._*
# Files that might appear in the root of a volume
.DocumentRevisions-V100
.fseventsd
.Spotlight-V100
.TemporaryItems
.Trashes
.VolumeIcon.icns
# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk

19
.travis.yml Normal file
View File

@ -0,0 +1,19 @@
language: python
python:
- "3.2"
- "3.3"
- "3.4"
- "3.5"
- "3.5-dev" # 3.5 development branch
- "3.6-dev" # 3.6 development branch
- "nightly" # currently points to 3.7-dev
# command to install dependencies
install: "pip install -r requirements.txt"
# command to run tests
#script: nosetests
#script: cd comic_dl
script:
- cd anime_dl
notifications:
email:
- xonshiz@psychoticelites.com

3
Changelog.md Normal file
View File

@ -0,0 +1,3 @@
#Changelog
- Site support for Crunchyroll.com [2017.03.05]

22
LICENSE Normal file
View File

@ -0,0 +1,22 @@
The MIT License (MIT)
Copyright (c) 2013-2016 Blackrock Digital LLC.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

140
ReadMe.md Normal file
View File

@ -0,0 +1,140 @@
# Anime-DL | [![Build Status](https://travis-ci.org/Xonshiz/anime-dl.svg?branch=master)](https://travis-ci.org/Xonshiz/anime-dl) [![Documentation Status](https://readthedocs.org/projects/anime-dl/badge/?version=latest)](http://anime-dl.readthedocs.io/en/latest/?badge=latest) | [![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.me/xonshiz)
Anime-dl is a Command-line program to download anime from CrunchyRoll and Funimation. This script needs you to have a premium subscription to the listed services. If you don't have a subscription, this script is pretty much usless for you.
> Downloading and distributing this content may be illegal.This script was written for education purposes purely and you are responsible for its use.
> Support these anime streaming websites by buying a premium account.
## Table of Content
* [Supported Sites](https://github.com/Xonshiz/anime-dl/blob/master/Supported_Sites.md)
* [Dependencies Installation](#dependencies-installation)
* [Installation](#installation)
* [Python Support](#python-support)
* [Windows Binary](#windows-binary)
* [List of Arguments](#list-of-arguments)
* [Usage](#usage)
* [Windows](#windows)
* [Linux/Debian](#linuxdebian)
* [Features](#features)
* [Changelog](https://github.com/Xonshiz/anime-dl/blob/master/Changelog.md)
* [Opening An Issue/Requesting A Site](#opening-an-issuerequesting-a-site)
* [Reporting Issues](#reporting-issues)
* [Suggesting A Feature](#suggesting-a-feature)
* [Donations](#donations)
## Supported Websites
You can check the list of supported websites [**`HERE`**](https://github.com/Xonshiz/anime-dl/blob/master/Supported_Sites.md).
## Dependencies Installation
This script can run on multiple Operating Systems. So, if you're using the `python` script instead of the `windows binary` of this script, then you'll need to get things ready first. Since this script doesn't rely on a lot of external dependencies, you just need to grab a few things, same for all operating systems.
1.) Make sure you have Python installed and is present in your system's path.
2.) Grab [FFmpeg from this link.](https://ffmpeg.org/download.html)
3.) Install FFmpeg and place it in the directory of this script, or put FFmpeg in your system's path.
4.) Browse to the directory of this script and open command prompt/shell in that directory and run this command :
```
python pip install -r requirements.txt
```
## Installation
After installing and setting up all the dependencies in your Operating System, you're good to go and use this script.
The instructions for all the OS would remain same. Download [`THIS REPOSITORY`](https://github.com/Xonshiz/anime-dl/archive/master.zip) and put it somewhere in your system. Move over the `anime_dl` folder.
**Windows users**, it's better to not place it places where it requires administrator privileges. Good example would be `C:\Windows`. This goes for both, the Python script and the windows binary file (.exe).
**Linux/Debian** users make sure that this script is executable.just run this command, if you run into problem(s) :
`chmod +x anime-dl.py`
and then, execute with this :
`./anime-dl.py`
## Python Support
This script supports only Pythom 3 currently..
## Windows Binary
It is recommended that windows users use this binary to save both, your head and time from installing all the dependencies.
You need to download [FFmpeg](https://ffmpeg.org/download.html) and keep it in the same directory as that of this windows binary file or you need to have PhantomJS in your path.
If you already have it, then you can download this binary and start using the script right off the bat :
* `Binary (x86)` : [Click Here](https://github.com/Xonshiz/anime-dl/releases/latest)
## List of Arguments
Currently, the script supports these arguments :
```
-h, --help Prints the basic help menu of the script and exits.
-i,--input Defines the input link to the anime.
-V,--version Prints the VERSION and exits.
-u,--username Indicates username for a website.
-p,--password Indicates password for a website.
-r,--resolution Indicates the desired resolution (default = 720p)
```
## Usage
With this script, you have to pass arguments in order to be able to download anything. Passing arguments in a script is pretty easy. Since the script is pretty basic, it doesn't have too many arguments. Go check the [`ARGUMENTS SECTION`](https://github.com/Xonshiz/anime-dl#list-of-arguments) to know more about which arguments the script offers.
Follow the instructions according to your OS :
### Windows
After you've saved this script in a directory/folder, you need to open `command prompt` and browse to that directory and then execute the script. Let's do it step by step :
* Open the folder where you've downloaded the files of this repository.
* Hold down the **`SHIFT`** key and while holding down the SHIFT key, **`RIGHT CLICK`** and select `Open Command Prompy Here` from the options that show up.
* Now, in the command prompt, type this :
*If you're using the windows binary :*
`anime-dl.exe -i "<URL TO THE ANIME>" -u "YourUsername" -p "Password" -r "Resolution"`
*If you're using the Python Script :*
`anime-dl.py -i "<URL TO THE ANIME>" -u "YourUsername" -p "Password" -r "Resolution"`
URL can be any URL of the [supported websites](https://github.com/Xonshiz/anime-dl/blob/master/Supported_Sites.md).
### Linux/Debian
After you've saved this script in a directory/folder, you need to open `command prompt` and browse to that directory and then execute the script. Let's do it step by step :
* Open a terminal, `Ctrl + Alt + T` is the shortcut to do so (if you didn't know).
* Now, change the current working directory of the terminal to the one where you've downloaded this repository.
* Now, in the Terminal, type this :
`anime-dl.py -i "<URL TO THE ANIME>" -u "YourUsername" -p "Password" -r "Resolution"`
URL can be any URL of the [supported websites](https://github.com/Xonshiz/anime-dl/blob/master/Supported_Sites.md).
## Features
This is a very basic and small sript, so at the moment it only have a few features.
* Downloads a Single episode along with all the available subtitles for that episode.
* Downloads and puts them all in a directory named "Output".
* Skip if the file has already been downloaded.
## Changelog
You can check the changelog [**`HERE`**](https://github.com/Xonshiz/anime-dl/blob/master/Changelog.md).
## Opening An Issue/Requesting A Site
If your're planning to open an issue for the script or ask for a new feature or anything that requires opening an Issue, then please do keep these things in mind.
### Reporting Issues
Please do make sure that you read and follow this :
### Suggesting A Feature
If you're here to make suggestions, please follow the basic syntax to post a request :
**Subject** : Something that briefly tells us about the feature.
**Long Explanation** : Describe in details what you want and how you want.
This should be enough, but it'll be great if you can add more ;)
# Donations
You can always send some money over from this :
Paypal : [![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.me/xonshiz)
Patreon Link : https://www.patreon.com/xonshiz
Any amount is appreciated :)

3
Supported_Sites.md Normal file
View File

@ -0,0 +1,3 @@
#List of Supported Websites
* [CrunchyRoll](http://crunchyroll.com)

45
anime_dl/AnimeDL.py Normal file
View File

@ -0,0 +1,45 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# from urllib.parse import urlparse
try:
from urllib.parse import urlparse
except ImportError:
from urlparse import urlparse
import anime_dl.sites
from re import match
class AnimeDL(object):
def __init__(self, url, username, password, resolution, language):
website = str(self.honcho(url=url[0]))
if website == "Crunchyroll":
if not url[0] or not username[0] or not password[0]:
print("Please enter the required arguments. Run __main__.py --help")
exit()
else:
Crunchy_Show_regex = r'https?://(?:(?P<prefix>www|m)\.)?(?P<url>crunchyroll\.com/(?!(?:news|anime-news|library|forum|launchcalendar|lineup|store|comics|freetrial|login))(?P<id>[\w\-]+))/?(?:\?|$)'
Crunchy_Video_regex = r'https?:\/\/(?:(?P<prefix>www|m)\.)?(?P<url>crunchyroll\.(?:com|fr)/(?:media(?:-|/\?id=)|[^/]*/[^/?&]*?)(?P<video_id>[0-9]+))(?:[/?&]|$)'
Crunchy_Show = match(Crunchy_Show_regex, url[0])
Crunchy_Video = match(Crunchy_Video_regex, url[0])
if Crunchy_Video:
anime_dl.sites.crunchyroll.CrunchyRoll(
url=url[0], password=password, username=username, resolution=resolution)
elif Crunchy_Show:
anime_dl.sites.crunchywhole.crunchyRollWhole()
def honcho(self, url):
# print("Got url : %s" % url)
# Verify that we have a sane url and return which website it belongs
# to.
domain = urlparse(url).netloc
# print(domain)
if domain in ["www.funimation.com", "funimation.com"]:
anime_dl.sites.funimation.Funimation()
elif domain in ["www.crunchyroll.com", "crunchyroll.com"]:
return "Crunchyroll"

2
anime_dl/__init__.py Normal file
View File

@ -0,0 +1,2 @@
import anime_dl.sites
import anime_dl.external

47
anime_dl/__main__.py Normal file
View File

@ -0,0 +1,47 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
__author__ = "Xonshiz"
__email__ = "xonshiz@psychoticelites.com"
"""
import anime_dl.AnimeDL
# import sys
from anime_dl.version import __version__
import argparse
class main(object):
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Anime_DL downloads anime from CrunchyRoll and Funimation.')
parser.add_argument('--version', action='store_true', help='Shows version and exits.')
required_args = parser.add_argument_group('Required Arguments :')
required_args.add_argument('-p', '--password', nargs=1, help='Indicates password for a website.')
required_args.add_argument('-u', '--username', nargs=1, help='Indicates username for a website.')
required_args.add_argument('-i', '--input', nargs=1, help='Inputs the URL to anime.')
parser.add_argument('-r', '--resolution', nargs=1, help='Inputs the URL to anime.', default='720p')
parser.add_argument('-l', '--language', nargs=1, help='Inputs the URL to anime.', default='Japanese')
args = parser.parse_args()
if args.version:
print("Current Version : %s" % __version__)
exit()
if args.username == None or args.password == None or args.input == None:
print("Please enter the required arguments. Run __main__.py --help")
exit()
else:
# If the argument has been provided for resolution and language, they're going to be lists, otherwise, they're
# going to be simple value == 720p.
# So, if return type comes out to be list, send the first element, otherwise, send 720p as it is.
# Same approach for the audio as well.
if type(args.resolution) == list:
args.resolution = args.resolution[0]
if type(args.language) == list:
args.language = args.language[0]
anime_dl.AnimeDL.AnimeDL(url= args.input, username=args.username, password=args.password, resolution=args.resolution, language=args.language)

4
anime_dl/external/__init__.py vendored Normal file
View File

@ -0,0 +1,4 @@
import anime_dl.external.aes
import anime_dl.external.compat
import anime_dl.external.socks
import anime_dl.external.utils

333
anime_dl/external/aes.py vendored Normal file
View File

@ -0,0 +1,333 @@
from __future__ import unicode_literals
import base64
from math import ceil
from .utils import bytes_to_intlist, intlist_to_bytes
# from utils import bytes_to_intlist, intlist_to_bytes
BLOCK_SIZE_BYTES = 16
def aes_ctr_decrypt(data, key, counter):
"""
Decrypt with aes in counter mode
@param {int[]} data cipher
@param {int[]} key 16/24/32-Byte cipher key
@param {instance} counter Instance whose next_value function (@returns {int[]} 16-Byte block)
returns the next counter block
@returns {int[]} decrypted data
"""
expanded_key = key_expansion(key)
block_count = int(ceil(float(len(data)) / BLOCK_SIZE_BYTES))
decrypted_data = []
for i in range(block_count):
counter_block = counter.next_value()
block = data[i * BLOCK_SIZE_BYTES: (i + 1) * BLOCK_SIZE_BYTES]
block += [0] * (BLOCK_SIZE_BYTES - len(block))
cipher_counter_block = aes_encrypt(counter_block, expanded_key)
decrypted_data += xor(block, cipher_counter_block)
decrypted_data = decrypted_data[:len(data)]
return decrypted_data
def aes_cbc_decrypt(data, key, iv):
"""
Decrypt with aes in CBC mode
@param {int[]} data cipher
@param {int[]} key 16/24/32-Byte cipher key
@param {int[]} iv 16-Byte IV
@returns {int[]} decrypted data
"""
expanded_key = key_expansion(key)
block_count = int(ceil(float(len(data)) / BLOCK_SIZE_BYTES))
decrypted_data = []
previous_cipher_block = iv
for i in range(block_count):
block = data[i * BLOCK_SIZE_BYTES: (i + 1) * BLOCK_SIZE_BYTES]
block += [0] * (BLOCK_SIZE_BYTES - len(block))
decrypted_block = aes_decrypt(block, expanded_key)
decrypted_data += xor(decrypted_block, previous_cipher_block)
previous_cipher_block = block
decrypted_data = decrypted_data[:len(data)]
return decrypted_data
def key_expansion(data):
"""
Generate key schedule
@param {int[]} data 16/24/32-Byte cipher key
@returns {int[]} 176/208/240-Byte expanded key
"""
data = data[:] # copy
rcon_iteration = 1
key_size_bytes = len(data)
expanded_key_size_bytes = (key_size_bytes // 4 + 7) * BLOCK_SIZE_BYTES
while len(data) < expanded_key_size_bytes:
temp = data[-4:]
temp = key_schedule_core(temp, rcon_iteration)
rcon_iteration += 1
data += xor(temp, data[-key_size_bytes: 4 - key_size_bytes])
for _ in range(3):
temp = data[-4:]
data += xor(temp, data[-key_size_bytes: 4 - key_size_bytes])
if key_size_bytes == 32:
temp = data[-4:]
temp = sub_bytes(temp)
data += xor(temp, data[-key_size_bytes: 4 - key_size_bytes])
for _ in range(3 if key_size_bytes == 32 else 2 if key_size_bytes == 24 else 0):
temp = data[-4:]
data += xor(temp, data[-key_size_bytes: 4 - key_size_bytes])
data = data[:expanded_key_size_bytes]
return data
def aes_encrypt(data, expanded_key):
"""
Encrypt one block with aes
@param {int[]} data 16-Byte state
@param {int[]} expanded_key 176/208/240-Byte expanded key
@returns {int[]} 16-Byte cipher
"""
rounds = len(expanded_key) // BLOCK_SIZE_BYTES - 1
data = xor(data, expanded_key[:BLOCK_SIZE_BYTES])
for i in range(1, rounds + 1):
data = sub_bytes(data)
data = shift_rows(data)
if i != rounds:
data = mix_columns(data)
data = xor(data, expanded_key[i * BLOCK_SIZE_BYTES: (i + 1) * BLOCK_SIZE_BYTES])
return data
def aes_decrypt(data, expanded_key):
"""
Decrypt one block with aes
@param {int[]} data 16-Byte cipher
@param {int[]} expanded_key 176/208/240-Byte expanded key
@returns {int[]} 16-Byte state
"""
rounds = len(expanded_key) // BLOCK_SIZE_BYTES - 1
for i in range(rounds, 0, -1):
data = xor(data, expanded_key[i * BLOCK_SIZE_BYTES: (i + 1) * BLOCK_SIZE_BYTES])
if i != rounds:
data = mix_columns_inv(data)
data = shift_rows_inv(data)
data = sub_bytes_inv(data)
data = xor(data, expanded_key[:BLOCK_SIZE_BYTES])
return data
def aes_decrypt_text(data, password, key_size_bytes):
"""
Decrypt text
- The first 8 Bytes of decoded 'data' are the 8 high Bytes of the counter
- The cipher key is retrieved by encrypting the first 16 Byte of 'password'
with the first 'key_size_bytes' Bytes from 'password' (if necessary filled with 0's)
- Mode of operation is 'counter'
@param {str} data Base64 encoded string
@param {str,unicode} password Password (will be encoded with utf-8)
@param {int} key_size_bytes Possible values: 16 for 128-Bit, 24 for 192-Bit or 32 for 256-Bit
@returns {str} Decrypted data
"""
NONCE_LENGTH_BYTES = 8
data = bytes_to_intlist(base64.b64decode(data.encode('utf-8')))
password = bytes_to_intlist(password.encode('utf-8'))
key = password[:key_size_bytes] + [0] * (key_size_bytes - len(password))
key = aes_encrypt(key[:BLOCK_SIZE_BYTES], key_expansion(key)) * (key_size_bytes // BLOCK_SIZE_BYTES)
nonce = data[:NONCE_LENGTH_BYTES]
cipher = data[NONCE_LENGTH_BYTES:]
class Counter(object):
__value = nonce + [0] * (BLOCK_SIZE_BYTES - NONCE_LENGTH_BYTES)
def next_value(self):
temp = self.__value
self.__value = inc(self.__value)
return temp
decrypted_data = aes_ctr_decrypt(cipher, key, Counter())
plaintext = intlist_to_bytes(decrypted_data)
return plaintext
RCON = (0x8d, 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x1b, 0x36)
SBOX = (0x63, 0x7C, 0x77, 0x7B, 0xF2, 0x6B, 0x6F, 0xC5, 0x30, 0x01, 0x67, 0x2B, 0xFE, 0xD7, 0xAB, 0x76,
0xCA, 0x82, 0xC9, 0x7D, 0xFA, 0x59, 0x47, 0xF0, 0xAD, 0xD4, 0xA2, 0xAF, 0x9C, 0xA4, 0x72, 0xC0,
0xB7, 0xFD, 0x93, 0x26, 0x36, 0x3F, 0xF7, 0xCC, 0x34, 0xA5, 0xE5, 0xF1, 0x71, 0xD8, 0x31, 0x15,
0x04, 0xC7, 0x23, 0xC3, 0x18, 0x96, 0x05, 0x9A, 0x07, 0x12, 0x80, 0xE2, 0xEB, 0x27, 0xB2, 0x75,
0x09, 0x83, 0x2C, 0x1A, 0x1B, 0x6E, 0x5A, 0xA0, 0x52, 0x3B, 0xD6, 0xB3, 0x29, 0xE3, 0x2F, 0x84,
0x53, 0xD1, 0x00, 0xED, 0x20, 0xFC, 0xB1, 0x5B, 0x6A, 0xCB, 0xBE, 0x39, 0x4A, 0x4C, 0x58, 0xCF,
0xD0, 0xEF, 0xAA, 0xFB, 0x43, 0x4D, 0x33, 0x85, 0x45, 0xF9, 0x02, 0x7F, 0x50, 0x3C, 0x9F, 0xA8,
0x51, 0xA3, 0x40, 0x8F, 0x92, 0x9D, 0x38, 0xF5, 0xBC, 0xB6, 0xDA, 0x21, 0x10, 0xFF, 0xF3, 0xD2,
0xCD, 0x0C, 0x13, 0xEC, 0x5F, 0x97, 0x44, 0x17, 0xC4, 0xA7, 0x7E, 0x3D, 0x64, 0x5D, 0x19, 0x73,
0x60, 0x81, 0x4F, 0xDC, 0x22, 0x2A, 0x90, 0x88, 0x46, 0xEE, 0xB8, 0x14, 0xDE, 0x5E, 0x0B, 0xDB,
0xE0, 0x32, 0x3A, 0x0A, 0x49, 0x06, 0x24, 0x5C, 0xC2, 0xD3, 0xAC, 0x62, 0x91, 0x95, 0xE4, 0x79,
0xE7, 0xC8, 0x37, 0x6D, 0x8D, 0xD5, 0x4E, 0xA9, 0x6C, 0x56, 0xF4, 0xEA, 0x65, 0x7A, 0xAE, 0x08,
0xBA, 0x78, 0x25, 0x2E, 0x1C, 0xA6, 0xB4, 0xC6, 0xE8, 0xDD, 0x74, 0x1F, 0x4B, 0xBD, 0x8B, 0x8A,
0x70, 0x3E, 0xB5, 0x66, 0x48, 0x03, 0xF6, 0x0E, 0x61, 0x35, 0x57, 0xB9, 0x86, 0xC1, 0x1D, 0x9E,
0xE1, 0xF8, 0x98, 0x11, 0x69, 0xD9, 0x8E, 0x94, 0x9B, 0x1E, 0x87, 0xE9, 0xCE, 0x55, 0x28, 0xDF,
0x8C, 0xA1, 0x89, 0x0D, 0xBF, 0xE6, 0x42, 0x68, 0x41, 0x99, 0x2D, 0x0F, 0xB0, 0x54, 0xBB, 0x16)
SBOX_INV = (0x52, 0x09, 0x6a, 0xd5, 0x30, 0x36, 0xa5, 0x38, 0xbf, 0x40, 0xa3, 0x9e, 0x81, 0xf3, 0xd7, 0xfb,
0x7c, 0xe3, 0x39, 0x82, 0x9b, 0x2f, 0xff, 0x87, 0x34, 0x8e, 0x43, 0x44, 0xc4, 0xde, 0xe9, 0xcb,
0x54, 0x7b, 0x94, 0x32, 0xa6, 0xc2, 0x23, 0x3d, 0xee, 0x4c, 0x95, 0x0b, 0x42, 0xfa, 0xc3, 0x4e,
0x08, 0x2e, 0xa1, 0x66, 0x28, 0xd9, 0x24, 0xb2, 0x76, 0x5b, 0xa2, 0x49, 0x6d, 0x8b, 0xd1, 0x25,
0x72, 0xf8, 0xf6, 0x64, 0x86, 0x68, 0x98, 0x16, 0xd4, 0xa4, 0x5c, 0xcc, 0x5d, 0x65, 0xb6, 0x92,
0x6c, 0x70, 0x48, 0x50, 0xfd, 0xed, 0xb9, 0xda, 0x5e, 0x15, 0x46, 0x57, 0xa7, 0x8d, 0x9d, 0x84,
0x90, 0xd8, 0xab, 0x00, 0x8c, 0xbc, 0xd3, 0x0a, 0xf7, 0xe4, 0x58, 0x05, 0xb8, 0xb3, 0x45, 0x06,
0xd0, 0x2c, 0x1e, 0x8f, 0xca, 0x3f, 0x0f, 0x02, 0xc1, 0xaf, 0xbd, 0x03, 0x01, 0x13, 0x8a, 0x6b,
0x3a, 0x91, 0x11, 0x41, 0x4f, 0x67, 0xdc, 0xea, 0x97, 0xf2, 0xcf, 0xce, 0xf0, 0xb4, 0xe6, 0x73,
0x96, 0xac, 0x74, 0x22, 0xe7, 0xad, 0x35, 0x85, 0xe2, 0xf9, 0x37, 0xe8, 0x1c, 0x75, 0xdf, 0x6e,
0x47, 0xf1, 0x1a, 0x71, 0x1d, 0x29, 0xc5, 0x89, 0x6f, 0xb7, 0x62, 0x0e, 0xaa, 0x18, 0xbe, 0x1b,
0xfc, 0x56, 0x3e, 0x4b, 0xc6, 0xd2, 0x79, 0x20, 0x9a, 0xdb, 0xc0, 0xfe, 0x78, 0xcd, 0x5a, 0xf4,
0x1f, 0xdd, 0xa8, 0x33, 0x88, 0x07, 0xc7, 0x31, 0xb1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xec, 0x5f,
0x60, 0x51, 0x7f, 0xa9, 0x19, 0xb5, 0x4a, 0x0d, 0x2d, 0xe5, 0x7a, 0x9f, 0x93, 0xc9, 0x9c, 0xef,
0xa0, 0xe0, 0x3b, 0x4d, 0xae, 0x2a, 0xf5, 0xb0, 0xc8, 0xeb, 0xbb, 0x3c, 0x83, 0x53, 0x99, 0x61,
0x17, 0x2b, 0x04, 0x7e, 0xba, 0x77, 0xd6, 0x26, 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d)
MIX_COLUMN_MATRIX = ((0x2, 0x3, 0x1, 0x1),
(0x1, 0x2, 0x3, 0x1),
(0x1, 0x1, 0x2, 0x3),
(0x3, 0x1, 0x1, 0x2))
MIX_COLUMN_MATRIX_INV = ((0xE, 0xB, 0xD, 0x9),
(0x9, 0xE, 0xB, 0xD),
(0xD, 0x9, 0xE, 0xB),
(0xB, 0xD, 0x9, 0xE))
RIJNDAEL_EXP_TABLE = (0x01, 0x03, 0x05, 0x0F, 0x11, 0x33, 0x55, 0xFF, 0x1A, 0x2E, 0x72, 0x96, 0xA1, 0xF8, 0x13, 0x35,
0x5F, 0xE1, 0x38, 0x48, 0xD8, 0x73, 0x95, 0xA4, 0xF7, 0x02, 0x06, 0x0A, 0x1E, 0x22, 0x66, 0xAA,
0xE5, 0x34, 0x5C, 0xE4, 0x37, 0x59, 0xEB, 0x26, 0x6A, 0xBE, 0xD9, 0x70, 0x90, 0xAB, 0xE6, 0x31,
0x53, 0xF5, 0x04, 0x0C, 0x14, 0x3C, 0x44, 0xCC, 0x4F, 0xD1, 0x68, 0xB8, 0xD3, 0x6E, 0xB2, 0xCD,
0x4C, 0xD4, 0x67, 0xA9, 0xE0, 0x3B, 0x4D, 0xD7, 0x62, 0xA6, 0xF1, 0x08, 0x18, 0x28, 0x78, 0x88,
0x83, 0x9E, 0xB9, 0xD0, 0x6B, 0xBD, 0xDC, 0x7F, 0x81, 0x98, 0xB3, 0xCE, 0x49, 0xDB, 0x76, 0x9A,
0xB5, 0xC4, 0x57, 0xF9, 0x10, 0x30, 0x50, 0xF0, 0x0B, 0x1D, 0x27, 0x69, 0xBB, 0xD6, 0x61, 0xA3,
0xFE, 0x19, 0x2B, 0x7D, 0x87, 0x92, 0xAD, 0xEC, 0x2F, 0x71, 0x93, 0xAE, 0xE9, 0x20, 0x60, 0xA0,
0xFB, 0x16, 0x3A, 0x4E, 0xD2, 0x6D, 0xB7, 0xC2, 0x5D, 0xE7, 0x32, 0x56, 0xFA, 0x15, 0x3F, 0x41,
0xC3, 0x5E, 0xE2, 0x3D, 0x47, 0xC9, 0x40, 0xC0, 0x5B, 0xED, 0x2C, 0x74, 0x9C, 0xBF, 0xDA, 0x75,
0x9F, 0xBA, 0xD5, 0x64, 0xAC, 0xEF, 0x2A, 0x7E, 0x82, 0x9D, 0xBC, 0xDF, 0x7A, 0x8E, 0x89, 0x80,
0x9B, 0xB6, 0xC1, 0x58, 0xE8, 0x23, 0x65, 0xAF, 0xEA, 0x25, 0x6F, 0xB1, 0xC8, 0x43, 0xC5, 0x54,
0xFC, 0x1F, 0x21, 0x63, 0xA5, 0xF4, 0x07, 0x09, 0x1B, 0x2D, 0x77, 0x99, 0xB0, 0xCB, 0x46, 0xCA,
0x45, 0xCF, 0x4A, 0xDE, 0x79, 0x8B, 0x86, 0x91, 0xA8, 0xE3, 0x3E, 0x42, 0xC6, 0x51, 0xF3, 0x0E,
0x12, 0x36, 0x5A, 0xEE, 0x29, 0x7B, 0x8D, 0x8C, 0x8F, 0x8A, 0x85, 0x94, 0xA7, 0xF2, 0x0D, 0x17,
0x39, 0x4B, 0xDD, 0x7C, 0x84, 0x97, 0xA2, 0xFD, 0x1C, 0x24, 0x6C, 0xB4, 0xC7, 0x52, 0xF6, 0x01)
RIJNDAEL_LOG_TABLE = (0x00, 0x00, 0x19, 0x01, 0x32, 0x02, 0x1a, 0xc6, 0x4b, 0xc7, 0x1b, 0x68, 0x33, 0xee, 0xdf, 0x03,
0x64, 0x04, 0xe0, 0x0e, 0x34, 0x8d, 0x81, 0xef, 0x4c, 0x71, 0x08, 0xc8, 0xf8, 0x69, 0x1c, 0xc1,
0x7d, 0xc2, 0x1d, 0xb5, 0xf9, 0xb9, 0x27, 0x6a, 0x4d, 0xe4, 0xa6, 0x72, 0x9a, 0xc9, 0x09, 0x78,
0x65, 0x2f, 0x8a, 0x05, 0x21, 0x0f, 0xe1, 0x24, 0x12, 0xf0, 0x82, 0x45, 0x35, 0x93, 0xda, 0x8e,
0x96, 0x8f, 0xdb, 0xbd, 0x36, 0xd0, 0xce, 0x94, 0x13, 0x5c, 0xd2, 0xf1, 0x40, 0x46, 0x83, 0x38,
0x66, 0xdd, 0xfd, 0x30, 0xbf, 0x06, 0x8b, 0x62, 0xb3, 0x25, 0xe2, 0x98, 0x22, 0x88, 0x91, 0x10,
0x7e, 0x6e, 0x48, 0xc3, 0xa3, 0xb6, 0x1e, 0x42, 0x3a, 0x6b, 0x28, 0x54, 0xfa, 0x85, 0x3d, 0xba,
0x2b, 0x79, 0x0a, 0x15, 0x9b, 0x9f, 0x5e, 0xca, 0x4e, 0xd4, 0xac, 0xe5, 0xf3, 0x73, 0xa7, 0x57,
0xaf, 0x58, 0xa8, 0x50, 0xf4, 0xea, 0xd6, 0x74, 0x4f, 0xae, 0xe9, 0xd5, 0xe7, 0xe6, 0xad, 0xe8,
0x2c, 0xd7, 0x75, 0x7a, 0xeb, 0x16, 0x0b, 0xf5, 0x59, 0xcb, 0x5f, 0xb0, 0x9c, 0xa9, 0x51, 0xa0,
0x7f, 0x0c, 0xf6, 0x6f, 0x17, 0xc4, 0x49, 0xec, 0xd8, 0x43, 0x1f, 0x2d, 0xa4, 0x76, 0x7b, 0xb7,
0xcc, 0xbb, 0x3e, 0x5a, 0xfb, 0x60, 0xb1, 0x86, 0x3b, 0x52, 0xa1, 0x6c, 0xaa, 0x55, 0x29, 0x9d,
0x97, 0xb2, 0x87, 0x90, 0x61, 0xbe, 0xdc, 0xfc, 0xbc, 0x95, 0xcf, 0xcd, 0x37, 0x3f, 0x5b, 0xd1,
0x53, 0x39, 0x84, 0x3c, 0x41, 0xa2, 0x6d, 0x47, 0x14, 0x2a, 0x9e, 0x5d, 0x56, 0xf2, 0xd3, 0xab,
0x44, 0x11, 0x92, 0xd9, 0x23, 0x20, 0x2e, 0x89, 0xb4, 0x7c, 0xb8, 0x26, 0x77, 0x99, 0xe3, 0xa5,
0x67, 0x4a, 0xed, 0xde, 0xc5, 0x31, 0xfe, 0x18, 0x0d, 0x63, 0x8c, 0x80, 0xc0, 0xf7, 0x70, 0x07)
def sub_bytes(data):
return [SBOX[x] for x in data]
def sub_bytes_inv(data):
return [SBOX_INV[x] for x in data]
def rotate(data):
return data[1:] + [data[0]]
def key_schedule_core(data, rcon_iteration):
data = rotate(data)
data = sub_bytes(data)
data[0] = data[0] ^ RCON[rcon_iteration]
return data
def xor(data1, data2):
return [x ^ y for x, y in zip(data1, data2)]
def rijndael_mul(a, b):
if(a == 0 or b == 0):
return 0
return RIJNDAEL_EXP_TABLE[(RIJNDAEL_LOG_TABLE[a] + RIJNDAEL_LOG_TABLE[b]) % 0xFF]
def mix_column(data, matrix):
data_mixed = []
for row in range(4):
mixed = 0
for column in range(4):
# xor is (+) and (-)
mixed ^= rijndael_mul(data[column], matrix[row][column])
data_mixed.append(mixed)
return data_mixed
def mix_columns(data, matrix=MIX_COLUMN_MATRIX):
data_mixed = []
for i in range(4):
column = data[i * 4: (i + 1) * 4]
data_mixed += mix_column(column, matrix)
return data_mixed
def mix_columns_inv(data):
return mix_columns(data, MIX_COLUMN_MATRIX_INV)
def shift_rows(data):
data_shifted = []
for column in range(4):
for row in range(4):
data_shifted.append(data[((column + row) & 0b11) * 4 + row])
return data_shifted
def shift_rows_inv(data):
data_shifted = []
for column in range(4):
for row in range(4):
data_shifted.append(data[((column - row) & 0b11) * 4 + row])
return data_shifted
def inc(data):
data = data[:] # copy
for i in range(len(data) - 1, -1, -1):
if data[i] == 255:
data[i] = 0
else:
data[i] = data[i] + 1
break
return data
__all__ = ['aes_encrypt', 'key_expansion', 'aes_ctr_decrypt', 'aes_cbc_decrypt', 'aes_decrypt_text']

2931
anime_dl/external/compat.py vendored Normal file
View File

File diff suppressed because it is too large Load Diff

272
anime_dl/external/socks.py vendored Normal file
View File

@ -0,0 +1,272 @@
# Public Domain SOCKS proxy protocol implementation
# Adapted from https://gist.github.com/bluec0re/cafd3764412967417fd3
from __future__ import unicode_literals
# References:
# SOCKS4 protocol http://www.openssh.com/txt/socks4.protocol
# SOCKS4A protocol http://www.openssh.com/txt/socks4a.protocol
# SOCKS5 protocol https://tools.ietf.org/html/rfc1928
# SOCKS5 username/password authentication https://tools.ietf.org/html/rfc1929
import collections
import socket
from compat import (
compat_ord,
compat_struct_pack,
compat_struct_unpack,
)
__author__ = 'Timo Schmid <coding@timoschmid.de>'
SOCKS4_VERSION = 4
SOCKS4_REPLY_VERSION = 0x00
# Excerpt from SOCKS4A protocol:
# if the client cannot resolve the destination host's domain name to find its
# IP address, it should set the first three bytes of DSTIP to NULL and the last
# byte to a non-zero value.
SOCKS4_DEFAULT_DSTIP = compat_struct_pack('!BBBB', 0, 0, 0, 0xFF)
SOCKS5_VERSION = 5
SOCKS5_USER_AUTH_VERSION = 0x01
SOCKS5_USER_AUTH_SUCCESS = 0x00
class Socks4Command(object):
CMD_CONNECT = 0x01
CMD_BIND = 0x02
class Socks5Command(Socks4Command):
CMD_UDP_ASSOCIATE = 0x03
class Socks5Auth(object):
AUTH_NONE = 0x00
AUTH_GSSAPI = 0x01
AUTH_USER_PASS = 0x02
AUTH_NO_ACCEPTABLE = 0xFF # For server response
class Socks5AddressType(object):
ATYP_IPV4 = 0x01
ATYP_DOMAINNAME = 0x03
ATYP_IPV6 = 0x04
class ProxyError(socket.error):
ERR_SUCCESS = 0x00
def __init__(self, code=None, msg=None):
if code is not None and msg is None:
msg = self.CODES.get(code) or 'unknown error'
super(ProxyError, self).__init__(code, msg)
class InvalidVersionError(ProxyError):
def __init__(self, expected_version, got_version):
msg = ('Invalid response version from server. Expected {0:02x} got '
'{1:02x}'.format(expected_version, got_version))
super(InvalidVersionError, self).__init__(0, msg)
class Socks4Error(ProxyError):
ERR_SUCCESS = 90
CODES = {
91: 'request rejected or failed',
92: 'request rejected because SOCKS server cannot connect to identd on the client',
93: 'request rejected because the client program and identd report different user-ids'
}
class Socks5Error(ProxyError):
ERR_GENERAL_FAILURE = 0x01
CODES = {
0x01: 'general SOCKS server failure',
0x02: 'connection not allowed by ruleset',
0x03: 'Network unreachable',
0x04: 'Host unreachable',
0x05: 'Connection refused',
0x06: 'TTL expired',
0x07: 'Command not supported',
0x08: 'Address type not supported',
0xFE: 'unknown username or invalid password',
0xFF: 'all offered authentication methods were rejected'
}
class ProxyType(object):
SOCKS4 = 0
SOCKS4A = 1
SOCKS5 = 2
Proxy = collections.namedtuple('Proxy', (
'type', 'host', 'port', 'username', 'password', 'remote_dns'))
class sockssocket(socket.socket):
def __init__(self, *args, **kwargs):
self._proxy = None
super(sockssocket, self).__init__(*args, **kwargs)
def setproxy(self, proxytype, addr, port, rdns=True, username=None, password=None):
assert proxytype in (ProxyType.SOCKS4, ProxyType.SOCKS4A, ProxyType.SOCKS5)
self._proxy = Proxy(proxytype, addr, port, username, password, rdns)
def recvall(self, cnt):
data = b''
while len(data) < cnt:
cur = self.recv(cnt - len(data))
if not cur:
raise EOFError('{0} bytes missing'.format(cnt - len(data)))
data += cur
return data
def _recv_bytes(self, cnt):
data = self.recvall(cnt)
return compat_struct_unpack('!{0}B'.format(cnt), data)
@staticmethod
def _len_and_data(data):
return compat_struct_pack('!B', len(data)) + data
def _check_response_version(self, expected_version, got_version):
if got_version != expected_version:
self.close()
raise InvalidVersionError(expected_version, got_version)
def _resolve_address(self, destaddr, default, use_remote_dns):
try:
return socket.inet_aton(destaddr)
except socket.error:
if use_remote_dns and self._proxy.remote_dns:
return default
else:
return socket.inet_aton(socket.gethostbyname(destaddr))
def _setup_socks4(self, address, is_4a=False):
destaddr, port = address
ipaddr = self._resolve_address(destaddr, SOCKS4_DEFAULT_DSTIP, use_remote_dns=is_4a)
packet = compat_struct_pack('!BBH', SOCKS4_VERSION, Socks4Command.CMD_CONNECT, port) + ipaddr
username = (self._proxy.username or '').encode('utf-8')
packet += username + b'\x00'
if is_4a and self._proxy.remote_dns:
packet += destaddr.encode('utf-8') + b'\x00'
self.sendall(packet)
version, resp_code, dstport, dsthost = compat_struct_unpack('!BBHI', self.recvall(8))
self._check_response_version(SOCKS4_REPLY_VERSION, version)
if resp_code != Socks4Error.ERR_SUCCESS:
self.close()
raise Socks4Error(resp_code)
return (dsthost, dstport)
def _setup_socks4a(self, address):
self._setup_socks4(address, is_4a=True)
def _socks5_auth(self):
packet = compat_struct_pack('!B', SOCKS5_VERSION)
auth_methods = [Socks5Auth.AUTH_NONE]
if self._proxy.username and self._proxy.password:
auth_methods.append(Socks5Auth.AUTH_USER_PASS)
packet += compat_struct_pack('!B', len(auth_methods))
packet += compat_struct_pack('!{0}B'.format(len(auth_methods)), *auth_methods)
self.sendall(packet)
version, method = self._recv_bytes(2)
self._check_response_version(SOCKS5_VERSION, version)
if method == Socks5Auth.AUTH_NO_ACCEPTABLE:
self.close()
raise Socks5Error(method)
if method == Socks5Auth.AUTH_USER_PASS:
username = self._proxy.username.encode('utf-8')
password = self._proxy.password.encode('utf-8')
packet = compat_struct_pack('!B', SOCKS5_USER_AUTH_VERSION)
packet += self._len_and_data(username) + self._len_and_data(password)
self.sendall(packet)
version, status = self._recv_bytes(2)
self._check_response_version(SOCKS5_USER_AUTH_VERSION, version)
if status != SOCKS5_USER_AUTH_SUCCESS:
self.close()
raise Socks5Error(Socks5Error.ERR_GENERAL_FAILURE)
def _setup_socks5(self, address):
destaddr, port = address
ipaddr = self._resolve_address(destaddr, None, use_remote_dns=True)
self._socks5_auth()
reserved = 0
packet = compat_struct_pack('!BBB', SOCKS5_VERSION, Socks5Command.CMD_CONNECT, reserved)
if ipaddr is None:
destaddr = destaddr.encode('utf-8')
packet += compat_struct_pack('!B', Socks5AddressType.ATYP_DOMAINNAME)
packet += self._len_and_data(destaddr)
else:
packet += compat_struct_pack('!B', Socks5AddressType.ATYP_IPV4) + ipaddr
packet += compat_struct_pack('!H', port)
self.sendall(packet)
version, status, reserved, atype = self._recv_bytes(4)
self._check_response_version(SOCKS5_VERSION, version)
if status != Socks5Error.ERR_SUCCESS:
self.close()
raise Socks5Error(status)
if atype == Socks5AddressType.ATYP_IPV4:
destaddr = self.recvall(4)
elif atype == Socks5AddressType.ATYP_DOMAINNAME:
alen = compat_ord(self.recv(1))
destaddr = self.recvall(alen)
elif atype == Socks5AddressType.ATYP_IPV6:
destaddr = self.recvall(16)
destport = compat_struct_unpack('!H', self.recvall(2))[0]
return (destaddr, destport)
def _make_proxy(self, connect_func, address):
if not self._proxy:
return connect_func(self, address)
result = connect_func(self, (self._proxy.host, self._proxy.port))
if result != 0 and result is not None:
return result
setup_funcs = {
ProxyType.SOCKS4: self._setup_socks4,
ProxyType.SOCKS4A: self._setup_socks4a,
ProxyType.SOCKS5: self._setup_socks5,
}
setup_funcs[self._proxy.type](address)
return result
def connect(self, address):
self._make_proxy(socket.socket.connect, address)
def connect_ex(self, address):
return self._make_proxy(socket.socket.connect_ex, address)

3577
anime_dl/external/utils.py vendored Normal file
View File

File diff suppressed because it is too large Load Diff

3
anime_dl/sites/__init__.py Normal file
View File

@ -0,0 +1,3 @@
import anime_dl.sites.crunchyroll
import anime_dl.sites.crunchywhole
import anime_dl.sites.funimation

486
anime_dl/sites/crunchyroll.py Normal file
View File

@ -0,0 +1,486 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from cfscrape import create_scraper
from requests import session
from re import search, findall
from os import getcwd
from subprocess import call
# External libs have been taken from youtube-dl for decoding the subtitles.
from anime_dl.external.utils import bytes_to_intlist, intlist_to_bytes
from anime_dl.external.aes import aes_cbc_decrypt
from anime_dl.external.compat import compat_etree_fromstring
import zlib
import base64
from hashlib import sha1
from math import pow, sqrt, floor
from os import path, makedirs
from glob import glob
from shutil import move
class CrunchyRoll(object):
def __init__(self, url, password, username, resolution):
# cookies + Token are required to login after CR put their login page behind CloudFlare.
cookies, Token = self.webpagedownloader(url=url)
self.singleEpisode(
url=url, cookies=cookies, token=Token, resolution=resolution)
def loginCheck(self, htmlsource):
# Open the page and check the title. CrunchyRoll redirects the user and the title has the text "Redirecting...".
# If this is not found, you're probably not logged in and you'll just get 360p or 480p.
titleCheck = search(r'\<title\>(.*?)\</title\>',
str(htmlsource)).group(1)
if str(titleCheck) == "Redirecting...":
return True
else:
return False
def webpagedownloader(self, url):
headers = {
'User-Agent':
'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36',
'Referer':
'https://www.crunchyroll.com/login'
}
sess = session()
sess = create_scraper(sess)
print("Trying to login...")
initialPagefetch = sess.get(
url='https://www.crunchyroll.com/login', headers=headers).text
initialCookies = sess.cookies
csrfToken = search(r'login_form\[\_token\]\"\ value\=\"(.*?)\"',
str(initialPagefetch)).group(1)
# print(csrfToken)
payload = {
'login_form[name]': 'sharnitanjones@yahoo.com',
'login_form[password]': 'gemini617',
'login_form[redirect_url]': '/',
'login_form[_token]': '%s' % csrfToken
}
loginPost = sess.post(
url='https://www.crunchyroll.com/login',
data=payload,
headers=headers,
cookies=initialCookies)
if self.loginCheck(htmlsource=loginPost.text):
print("Logged in successfully...")
resp = sess.get(
url=url, headers=headers,
cookies=initialCookies).text.encode('utf-8')
# video_id = int(str(search(r'div\[media_id\=(.*?)\]', str(resp)).group(1)).strip())
#
return initialCookies, csrfToken
else:
print("Unable to Log you in. Check credentials again.")
def singleEpisode(self, url, cookies, token, resolution):
# print("Inside single episode")
current_directory = getcwd()
video_id = str(url.split('-')[-1]).replace("/", "")
# print("URL : %s\nCookies : %s\nToken : %s\nResolution : %s\nMedia ID : %s" % (url, cookies, token, resolution, video_id))
headers = {
'User-Agent':
'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36',
'Upgrade-Insecure-Requests':
'1',
'Accept-Encoding':
'gzip, deflate'
}
sess = session()
sess = create_scraper(sess)
if str(resolution).lower() in ['1080p', '1080', 'best', 'fhd']:
print("Grabbing Links for 1080p Streams.")
infoURL = "http://www.crunchyroll.com/xml/?req=RpcApiVideoPlayer_GetStandardConfig&media_id=%s&video_format=108&video_quality=80&current_page=%s" % (
video_id, url)
xml_page = sess.get(
url=infoURL, headers=headers, cookies=cookies).text
m3u8_link_raw = str(
search(r'\<file\>(.*?)\<\/file\>', xml_page).group(
1)).strip().replace("&amp;", "&")
anime_name = str(
search(r'\<series_title\>(.*?)\<\/series_title\>', xml_page)
.group(1)).strip().replace("’", "'").replace(
":", " - ").replace("&#039;", "'")
episode_number = str(
search(r'\<episode_number\>(.*?)\<\/episode_number\>',
xml_page).group(1)).strip()
width = str(
search(r'\<width\>(.*?)\<\/width\>', xml_page).group(
1)).strip()
height = str(
search(r'\<height\>(.*?)\<\/height\>', xml_page).group(
1)).strip()
# print("m3u8_link : %s\nanime_name : %s\nepisode_number : %s\nwidth : %s\nheight : %s\n" % (m3u8_link_raw, anime_name, episode_number, width, height))
# self.subFetcher(xml=str(xml_page), anime_name=anime_name, episode_number=episode_number)
file_name = str(anime_name) + " - " + str(
episode_number) + " [%sx%s].mp4" % (width, height)
# print("File Name : %s\n" % file_name)
if not path.exists("Output"):
makedirs("Output")
if path.isfile("Output/" + file_name):
print('[Anime-dl] File Exist! Skipping ', file_name, '\n')
pass
else:
self.subFetcher(
xml=str(xml_page),
anime_name=anime_name,
episode_number=episode_number)
# UNCOMMENT THIS LINE!!!
m3u8_file = sess.get(
url=m3u8_link_raw, cookies=cookies,
headers=headers).text.splitlines()[2]
# print("M3u8 : %s" % m3u8_file)
ffmpeg_command = "ffmpeg -i \"%s\" -c copy -bsf:a aac_adtstoasc \"%s\"" % (
m3u8_file, file_name)
call(ffmpeg_command)
for video_file in glob("*.mp4"):
try:
move(video_file, "Output")
except Exception as e:
print(str(e))
pass
for sub_files in glob("*.ass"):
try:
move(sub_files, "Output")
except Exception as e:
print(str(e))
pass
if str(resolution).lower() in ['720p', '720', 'hd']:
print("Grabbing Links for 720p Streams.")
infoURL = "http://www.crunchyroll.com/xml/?req=RpcApiVideoPlayer_GetStandardConfig&media_id=%s&video_format=106&video_quality=62&current_page=%s" % (
video_id, url)
xml_page = sess.get(
url=infoURL, headers=headers, cookies=cookies).text
m3u8_link_raw = str(
search(r'\<file\>(.*?)\<\/file\>', xml_page).group(
1)).strip().replace("&amp;", "&")
anime_name = str(
search(r'\<series_title\>(.*?)\<\/series_title\>', xml_page)
.group(1)).strip().replace("’", "'").replace(
":", " - ").replace("&#039;", "'")
episode_number = str(
search(r'\<episode_number\>(.*?)\<\/episode_number\>',
xml_page).group(1)).strip()
width = str(
search(r'\<width\>(.*?)\<\/width\>', xml_page).group(
1)).strip()
height = str(
search(r'\<height\>(.*?)\<\/height\>', xml_page).group(
1)).strip()
# print("m3u8_link : %s\nanime_name : %s\nepisode_number : %s\nwidth : %s\nheight : %s\n" % (m3u8_link_raw, anime_name, episode_number, width, height))
# self.subFetcher(xml=str(xml_page), anime_name=anime_name, episode_number=episode_number)
file_name = str(anime_name) + " - " + str(
episode_number) + " [%sx%s].mp4" % (width, height)
# print("File Name : %s\n" % file_name)
if not path.exists("Output"):
makedirs("Output")
if path.isfile("Output/" + file_name):
print('[Anime-dl] File Exist! Skipping %s\n' % file_name)
pass
else:
self.subFetcher(
xml=str(xml_page),
anime_name=anime_name,
episode_number=episode_number)
# UNCOMMENT THIS LINE!!!
m3u8_file = sess.get(
url=m3u8_link_raw, cookies=cookies,
headers=headers).text.splitlines()[2]
# print("M3u8 : %s" % m3u8_file)
ffmpeg_command = "ffmpeg -i \"%s\" -c copy -bsf:a aac_adtstoasc \"%s\"" % (
m3u8_file, file_name)
call(ffmpeg_command)
for video_file in glob("*.mp4"):
try:
move(video_file, "Output")
except Exception as e:
print(str(e))
pass
for sub_files in glob("*.ass"):
try:
move(sub_files, "Output")
except Exception as e:
print(str(e))
pass
if str(resolution).lower() in ['480p', '480', 'sd']:
print("Grabbing Links for 480p Streams.")
infoURL = "http://www.crunchyroll.com/xml/?req=RpcApiVideoPlayer_GetStandardConfig&media_id=%s&video_format=106&video_quality=61&current_page=%s" % (
video_id, url)
xml_page = sess.get(
url=infoURL, headers=headers, cookies=cookies).text
m3u8_link_raw = str(
search(r'\<file\>(.*?)\<\/file\>', xml_page).group(
1)).strip().replace("&amp;", "&")
anime_name = str(
search(r'\<series_title\>(.*?)\<\/series_title\>', xml_page)
.group(1)).strip().replace("’", "'").replace(
":", " - ").replace("&#039;", "'")
episode_number = str(
search(r'\<episode_number\>(.*?)\<\/episode_number\>',
xml_page).group(1)).strip()
width = str(
search(r'\<width\>(.*?)\<\/width\>', xml_page).group(
1)).strip()
height = str(
search(r'\<height\>(.*?)\<\/height\>', xml_page).group(
1)).strip()
# print("m3u8_link : %s\nanime_name : %s\nepisode_number : %s\nwidth : %s\nheight : %s\n" % (m3u8_link_raw, anime_name, episode_number, width, height))
# self.subFetcher(xml=str(xml_page), anime_name=anime_name, episode_number=episode_number)
file_name = str(anime_name) + " - " + str(
episode_number) + " [%sx%s].mp4" % (width, height)
# print("File Name : %s\n" % file_name)
if not path.exists("Output"):
makedirs("Output")
if path.isfile("Output/" + file_name):
print('[Anime-dl] File Exist! Skipping ', file_name, '\n')
pass
else:
self.subFetcher(
xml=str(xml_page),
anime_name=anime_name,
episode_number=episode_number)
# UNCOMMENT THIS LINE!!!
m3u8_file = sess.get(
url=m3u8_link_raw, cookies=cookies,
headers=headers).text.splitlines()[2]
# print("M3u8 : %s" % m3u8_file)
ffmpeg_command = "ffmpeg -i \"%s\" -c copy -bsf:a aac_adtstoasc \"%s\"" % (
m3u8_file, file_name)
call(ffmpeg_command)
for video_file in glob("*.mp4"):
try:
move(video_file, "Output")
except Exception as e:
print(str(e))
pass
for sub_files in glob("*.ass"):
try:
move(sub_files, "Output")
except Exception as e:
print(str(e))
pass
if str(resolution).lower() in ['360p', '360', 'mobile']:
print("Grabbing Links for 360p Streams.")
infoURL = "http://www.crunchyroll.com/xml/?req=RpcApiVideoPlayer_GetStandardConfig&media_id=%s&video_format=106&video_quality=60&current_page=%s" % (
video_id, url)
xml_page = sess.get(
url=infoURL, headers=headers, cookies=cookies).text
m3u8_link_raw = str(
search(r'\<file\>(.*?)\<\/file\>', xml_page).group(
1)).strip().replace("&amp;", "&")
anime_name = str(
search(r'\<series_title\>(.*?)\<\/series_title\>', xml_page)
.group(1)).strip().replace("’", "'").replace(
":", " - ").replace("&#039;", "'")
episode_number = str(
search(r'\<episode_number\>(.*?)\<\/episode_number\>',
xml_page).group(1)).strip()
width = str(
search(r'\<width\>(.*?)\<\/width\>', xml_page).group(
1)).strip()
height = str(
search(r'\<height\>(.*?)\<\/height\>', xml_page).group(
1)).strip()
# print("m3u8_link : %s\nanime_name : %s\nepisode_number : %s\nwidth : %s\nheight : %s\n" % (m3u8_link_raw, anime_name, episode_number, width, height))
# self.subFetcher(xml=str(xml_page), anime_name=anime_name, episode_number=episode_number)
file_name = str(anime_name) + " - " + str(
episode_number) + " [%sx%s].mp4" % (width, height)
# print("File Name : %s\n" % file_name)
if not path.exists("Output"):
makedirs("Output")
if path.isfile("Output/" + file_name):
print('[Anime-dl] File Exist! Skipping ', file_name, '\n')
pass
else:
self.subFetcher(
xml=str(xml_page),
anime_name=anime_name,
episode_number=episode_number)
# UNCOMMENT THIS LINE!!!
m3u8_file = sess.get(
url=m3u8_link_raw, cookies=cookies,
headers=headers).text.splitlines()[2]
# print("M3u8 : %s" % m3u8_file)
ffmpeg_command = "ffmpeg -i \"%s\" -c copy -bsf:a aac_adtstoasc \"%s\"" % (
m3u8_file, file_name)
call(ffmpeg_command)
for video_file in glob("*.mp4"):
try:
move(video_file, "Output")
except Exception as e:
print(str(e))
pass
for sub_files in glob("*.ass"):
try:
move(sub_files, "Output")
except Exception as e:
print(str(e))
pass
print("Completed Downloading : %s" % anime_name)
return (video_id, m3u8_link_raw, anime_name, episode_number, width,
height, file_name, cookies, token)
def subFetcher(self, xml, anime_name, episode_number):
headers = {
'User-Agent':
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/601.2.7 (KHTML, like Gecko) Version/9.0.1 Safari/601.2.7',
'Referer':
'https://www.crunchyroll.com'
}
sess = session()
sess = create_scraper(sess)
for sub_id, sub_lang, sub_lang2 in findall(
r'subtitle_script_id\=(.*?)\'\ title\=\'\[(.*?)\]\ (.*?)\'',
str(xml)):
# print("Sub ID : %s\t| Sub Lang : %s" % (sub_id, sub_lang))
xml_return = str(
sess.get(
url="http://www.crunchyroll.com/xml/?req=RpcApiSubtitle_GetXml&subtitle_script_id=%s"
% sub_id,
headers=headers).text)
# print(xml_return)
iv = str(
search(r'\<iv\>(.*?)\<\/iv\>', xml_return).group(1)).strip()
data = str(
search(r'\<data\>(.*?)\<\/data\>', xml_return).group(
1)).strip()
# print("Sub ID : %s\t| iv : %s\t| data : %s" % (sub_id, iv, data))
subtitle = self._decrypt_subtitles(data, iv,
sub_id).decode('utf-8')
# print(subtitle)
sub_root = compat_etree_fromstring(subtitle)
sub_data = self._convert_subtitles_to_ass(sub_root)
# print(sub_root)
lang_code = str(
search(r'lang_code\=\"(.*?)\"', str(subtitle)).group(
1)).strip()
sub_file_name = str(anime_name) + " - " + str(
episode_number) + ".%s.ass" % lang_code
# print(sub_file_name)
print("Writing subtitles to files...")
with open(sub_file_name, "w", encoding="utf-8") as sub_file:
sub_file.write(str(sub_data))
def _decrypt_subtitles(self, data, iv, id):
data = bytes_to_intlist(base64.b64decode(data.encode('utf-8')))
iv = bytes_to_intlist(base64.b64decode(iv.encode('utf-8')))
id = int(id)
def obfuscate_key_aux(count, modulo, start):
output = list(start)
for _ in range(count):
output.append(output[-1] + output[-2])
# cut off start values
output = output[2:]
output = list(map(lambda x: x % modulo + 33, output))
return output
def obfuscate_key(key):
num1 = int(floor(pow(2, 25) * sqrt(6.9)))
num2 = (num1 ^ key) << 5
num3 = key ^ num1
num4 = num3 ^ (num3 >> 3) ^ num2
prefix = intlist_to_bytes(obfuscate_key_aux(20, 97, (1, 2)))
shaHash = bytes_to_intlist(
sha1(prefix + str(num4).encode('ascii')).digest())
# Extend 160 Bit hash to 256 Bit
return shaHash + [0] * 12
key = obfuscate_key(id)
decrypted_data = intlist_to_bytes(aes_cbc_decrypt(data, key, iv))
return zlib.decompress(decrypted_data)
def _convert_subtitles_to_ass(self, sub_root):
output = ''
def ass_bool(strvalue):
assvalue = '0'
if strvalue == '1':
assvalue = '-1'
return assvalue
output = '[Script Info]\n'
output += 'Title: %s\n' % sub_root.attrib['title']
output += 'ScriptType: v4.00+\n'
output += 'WrapStyle: %s\n' % sub_root.attrib['wrap_style']
output += 'PlayResX: %s\n' % sub_root.attrib['play_res_x']
output += 'PlayResY: %s\n' % sub_root.attrib['play_res_y']
output += """
[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
"""
for style in sub_root.findall('./styles/style'):
output += 'Style: ' + style.attrib['name']
output += ',' + style.attrib['font_name']
output += ',' + style.attrib['font_size']
output += ',' + style.attrib['primary_colour']
output += ',' + style.attrib['secondary_colour']
output += ',' + style.attrib['outline_colour']
output += ',' + style.attrib['back_colour']
output += ',' + ass_bool(style.attrib['bold'])
output += ',' + ass_bool(style.attrib['italic'])
output += ',' + ass_bool(style.attrib['underline'])
output += ',' + ass_bool(style.attrib['strikeout'])
output += ',' + style.attrib['scale_x']
output += ',' + style.attrib['scale_y']
output += ',' + style.attrib['spacing']
output += ',' + style.attrib['angle']
output += ',' + style.attrib['border_style']
output += ',' + style.attrib['outline']
output += ',' + style.attrib['shadow']
output += ',' + style.attrib['alignment']
output += ',' + style.attrib['margin_l']
output += ',' + style.attrib['margin_r']
output += ',' + style.attrib['margin_v']
output += ',' + style.attrib['encoding']
output += '\n'
output += """
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
"""
for event in sub_root.findall('./events/event'):
output += 'Dialogue: 0'
output += ',' + event.attrib['start']
output += ',' + event.attrib['end']
output += ',' + event.attrib['style']
output += ',' + event.attrib['name']
output += ',' + event.attrib['margin_l']
output += ',' + event.attrib['margin_r']
output += ',' + event.attrib['margin_v']
output += ',' + event.attrib['effect']
output += ',' + event.attrib['text']
output += '\n'
return output

8
anime_dl/sites/crunchywhole.py Normal file
View File

@ -0,0 +1,8 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
class crunchyRollWhole(object):
def __init__(self):
print("This feature is only available for patrons.")
print("Visit : https://www.patreon.com/xonshiz")

8
anime_dl/sites/funimation.py Normal file
View File

@ -0,0 +1,8 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
class Funimation(object):
def __init__(self):
print("Under development!")
exit()

2
anime_dl/version.py Normal file
View File

@ -0,0 +1,2 @@
# Format : YY/MM/DD
__version__ = "2017.03.04"

3
docs/Changelog.md Normal file
View File

@ -0,0 +1,3 @@
#Changelog
- Site support for Crunchyroll.com [2017.03.05]

3
docs/Supported_Sites.md Normal file
View File

@ -0,0 +1,3 @@
#List of Supported Websites
* [CrunchyRoll](http://crunchyroll.com)

140
docs/index.md Normal file
View File

@ -0,0 +1,140 @@
# Anime-DL | [![Build Status](https://travis-ci.org/Xonshiz/anime-dl.svg?branch=master)](https://travis-ci.org/Xonshiz/anime-dl) [![Documentation Status](https://readthedocs.org/projects/anime-dl/badge/?version=latest)](http://anime-dl.readthedocs.io/en/latest/?badge=latest) | [![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.me/xonshiz)
Anime-dl is a Command-line program to download anime from CrunchyRoll and Funimation. This script needs you to have a premium subscription to the listed services. If you don't have a subscription, this script is pretty much usless for you.
> Downloading and distributing this content may be illegal.This script was written for education purposes purely and you are responsible for its use.
> Support these anime streaming websites by buying a premium account.
## Table of Content
* [Supported Sites](https://github.com/Xonshiz/anime-dl/blob/master/Supported_Sites.md)
* [Dependencies Installation](#dependencies-installation)
* [Installation](#installation)
* [Python Support](#python-support)
* [Windows Binary](#windows-binary)
* [List of Arguments](#list-of-arguments)
* [Usage](#usage)
* [Windows](#windows)
* [Linux/Debian](#linuxdebian)
* [Features](#features)
* [Changelog](https://github.com/Xonshiz/anime-dl/blob/master/Changelog.md)
* [Opening An Issue/Requesting A Site](#opening-an-issuerequesting-a-site)
* [Reporting Issues](#reporting-issues)
* [Suggesting A Feature](#suggesting-a-feature)
* [Donations](#donations)
## Supported Websites
You can check the list of supported websites [**`HERE`**](https://github.com/Xonshiz/anime-dl/blob/master/Supported_Sites.md).
## Dependencies Installation
This script can run on multiple Operating Systems. So, if you're using the `python` script instead of the `windows binary` of this script, then you'll need to get things ready first. Since this script doesn't rely on a lot of external dependencies, you just need to grab a few things, same for all operating systems.
1.) Make sure you have Python installed and is present in your system's path.
2.) Grab [FFmpeg from this link.](https://ffmpeg.org/download.html)
3.) Install FFmpeg and place it in the directory of this script, or put FFmpeg in your system's path.
4.) Browse to the directory of this script and open command prompt/shell in that directory and run this command :
```
python pip install -r requirements.txt
```
## Installation
After installing and setting up all the dependencies in your Operating System, you're good to go and use this script.
The instructions for all the OS would remain same. Download [`THIS REPOSITORY`](https://github.com/Xonshiz/anime-dl/archive/master.zip) and put it somewhere in your system. Move over the `anime_dl` folder.
**Windows users**, it's better to not place it places where it requires administrator privileges. Good example would be `C:\Windows`. This goes for both, the Python script and the windows binary file (.exe).
**Linux/Debian** users make sure that this script is executable.just run this command, if you run into problem(s) :
`chmod +x anime-dl.py`
and then, execute with this :
`./anime-dl.py`
## Python Support
This script supports only Pythom 3 currently..
## Windows Binary
It is recommended that windows users use this binary to save both, your head and time from installing all the dependencies.
You need to download [FFmpeg](https://ffmpeg.org/download.html) and keep it in the same directory as that of this windows binary file or you need to have PhantomJS in your path.
If you already have it, then you can download this binary and start using the script right off the bat :
* `Binary (x86)` : [Click Here](https://github.com/Xonshiz/anime-dl/releases/latest)
## List of Arguments
Currently, the script supports these arguments :
```
-h, --help Prints the basic help menu of the script and exits.
-i,--input Defines the input link to the anime.
-V,--version Prints the VERSION and exits.
-u,--username Indicates username for a website.
-p,--password Indicates password for a website.
-r,--resolution Indicates the desired resolution (default = 720p)
```
## Usage
With this script, you have to pass arguments in order to be able to download anything. Passing arguments in a script is pretty easy. Since the script is pretty basic, it doesn't have too many arguments. Go check the [`ARGUMENTS SECTION`](https://github.com/Xonshiz/anime-dl#list-of-arguments) to know more about which arguments the script offers.
Follow the instructions according to your OS :
### Windows
After you've saved this script in a directory/folder, you need to open `command prompt` and browse to that directory and then execute the script. Let's do it step by step :
* Open the folder where you've downloaded the files of this repository.
* Hold down the **`SHIFT`** key and while holding down the SHIFT key, **`RIGHT CLICK`** and select `Open Command Prompy Here` from the options that show up.
* Now, in the command prompt, type this :
*If you're using the windows binary :*
`anime-dl.exe -i "<URL TO THE ANIME>" -u "YourUsername" -p "Password" -r "Resolution"`
*If you're using the Python Script :*
`anime-dl.py -i "<URL TO THE ANIME>" -u "YourUsername" -p "Password" -r "Resolution"`
URL can be any URL of the [supported websites](https://github.com/Xonshiz/anime-dl/blob/master/Supported_Sites.md).
### Linux/Debian
After you've saved this script in a directory/folder, you need to open `command prompt` and browse to that directory and then execute the script. Let's do it step by step :
* Open a terminal, `Ctrl + Alt + T` is the shortcut to do so (if you didn't know).
* Now, change the current working directory of the terminal to the one where you've downloaded this repository.
* Now, in the Terminal, type this :
`anime-dl.py -i "<URL TO THE ANIME>" -u "YourUsername" -p "Password" -r "Resolution"`
URL can be any URL of the [supported websites](https://github.com/Xonshiz/anime-dl/blob/master/Supported_Sites.md).
## Features
This is a very basic and small sript, so at the moment it only have a few features.
* Downloads a Single episode along with all the available subtitles for that episode.
* Downloads and puts them all in a directory named "Output".
* Skip if the file has already been downloaded.
## Changelog
You can check the changelog [**`HERE`**](https://github.com/Xonshiz/anime-dl/blob/master/Changelog.md).
## Opening An Issue/Requesting A Site
If your're planning to open an issue for the script or ask for a new feature or anything that requires opening an Issue, then please do keep these things in mind.
### Reporting Issues
Please do make sure that you read and follow this :
### Suggesting A Feature
If you're here to make suggestions, please follow the basic syntax to post a request :
**Subject** : Something that briefly tells us about the feature.
**Long Explanation** : Describe in details what you want and how you want.
This should be enough, but it'll be great if you can add more ;)
# Donations
You can always send some money over from this :
Paypal : [![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.me/xonshiz)
Patreon Link : https://www.patreon.com/xonshiz
Any amount is appreciated :)

7
requirements.txt Normal file
View File

@ -0,0 +1,7 @@
argparse
hashlib
cfscrape
zlib
glob
shutil
base64