axcore
/
tartube
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add files via upload

master
A S Lewis 2019-05-28 14:48:25 +01:00 committed by GitHub
parent cdaa2ba53c
commit c1916c395a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
18 changed files with 1247 additions and 371 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

254
README.rst Normal file
View File

@ -0,0 +1,254 @@
Tartube
=======
Tartube is a GUI front-end for `youtube-dl <https://youtube-dl.org/>`__,
partly based on
`youtube-dl-gui <https://mrs0m30n3.github.io/youtube-dl-gui/>`__ and
written in Python 3 / Gtk 3.
Tartube is **alpha software**. It crashes a lot. If you find this
frustrating, find a solution and then `send it to
me <https://github.com/axcore/tartube/issues>`__.
Why should I use Tartube?
-------------------------
- You can download individual videos, and even whole channels and
playlists, from hundreds of different websites
- You can fetch information about those videos, channels and playlists,
without actually downloading anything
- Tartube will organise your videos into convenient folders
- Certain popular video websites manipulate search results, repeatedly
unsubscribe people from their favourite channels and/or deliberately
conceal videos which challenge the Californian political consensus.
Tartube won't do any of those things
- Tartube can, in some circumstances, see videos that are
region-blocked and age-restricted
Requirements
------------
- A working installation of `youtube-dl <https://youtube-dl.org/>`__
- `Python 3+ <https://www.python.org/downloads>`__
- `Gtk 3+ <https://python-gtk-3-tutorial.readthedocs.io/en/latest/>`__
- `Python validators module <https://pypi.org/project/validators/>`__
optional, but recommended
- `Python moviepy module <https://pypi.org/project/moviepy/>`__
optional
Downloads
---------
- `Source <http://tartube.sourceforge.io/>`__ from sourceforge.io
- `Source <https://github.com/axcore/tarbue>`__ from github
Installation
------------
Install from source
~~~~~~~~~~~~~~~~~~~
1. Download & extract the source
2. Change directory into the Tartube directory
3. Run ``python setup.py install``
Install using PyPI
~~~~~~~~~~~~~~~~~~
1. Run ``pip install tartube``
Install using MS Windows Installer
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
There is no installer for MS Windows yet.
Run without installing
~~~~~~~~~~~~~~~~~~~~~~
1. Download & extract the source
2. Change directory into the Tartube directory
3. Run 'python tartube.py'
Frequently-Asked Questions
--------------------------
**Q: I can't install Tartube!**
A: I have no experience of writing in Python and I'm still working it
out for myself. Contact me on the `Github
issues <https://github.com/axcore/tartube/issues>`__ page if you can do
better than me.
**Q: I can't run Tartube!**
A: See above.
**Q: Tartube doesn't work properly!**
A: See above.
**Q: Tartube keeps crashing!**
A: See above.
**Q: How do I use Tartube?**
A: Assuming that Tartube is installed and running correctly, then you
should start by checking that youtube-dl is also installed and running
correctly.
- Click **Operations > Update youtube-dl**
There are several locations on your filesystem where youtube-dl might
have been installed. If the update operation fails, you should try
modifying Tartube's settings.
- Click **Edit > System preferences...**
- Click the **youtube-dl** tab
- Try changing the setting **'Actual path to use during
download/update/refresh operations'**
- Try changing the setting **'Shell command for update operations'**
- Try the update operation again
On the left side of the Tartube window is a list of folders. You can
store videos, channels and playlists inside these folders. You can even
store folders inside of other folders.
Tartube saves videos on your filesystem using exactly the same
structure.
When you start Tartube, there are five folders already visible. You
can't remove any of these folders (but you can hide them, if you want).
Videos saved to the 'Temporary Videos' folder are deleted when Tartube
shuts down.
Once you've finished adding videos, channels, playlists and folders,
there are four things Tartube can do:
- **'Check'** - Fetch information about videos, but don't download them
- **'Download'** - Actually download the videos. If you have disabled
downloads for a particular item, Tartube will just fetch information
about it instead
- **'Update'** - Updates youtube-dl, as described above
- **'Refresh'** - Examines your filesystem. If you have manually copied
any videos into Tartube's data directory, those videos are added to
Tartube's database
**Protip:** Do an **'Update'** operation before you do a **'Check'** or
**'Download'** operation
**Protip:** Do a **'Check'** operation before you do **'Refresh'**
operation
youtube-dl offers a large number of download options. This is how to set
them.
- Click **Edit > General download options...**
Any changes you make in the new window aren't actually applied until you
click the **'Apply'** or **'OK'** buttons.
Those are the *default* download options. If you want to apply a
*different* set of download options to a particular channel or
particular playlist, you can do so.
For example, suppose you have added these folders and channels:
::
Comedy folder
CollegeHumor channel
PewDiePie channel
Politics folder
Liberal folder
The Young Turks channel
Conservative folder
Joe Rogan channel
Mark Dice channel
The general download options apply to all of these channels. Now,
suppose you apply some download options to the Politics folder:
- Right-click the folder, and select **Apply download options...**
Tartube's database now looks something like this:
::
Comedy folder
CollegeHumor channel
PewDiePie channel
++Politics folder
Liberal folder
The Young Turks channel
Conservative folder
Joe Rogan channel
Mark Dice channel
The new download options (marked ++) apply to *everything* inside the
Politics folder - The Young Turks, Joe Rogan and Mark Dice.
Now, suppose you add another set of download options (marked @@) to the
Conservative folder:
::
Comedy folder
CollegeHumor channel
PewDiePie channel
**Politics folder
Liberal folder
The Young Turks channel
@@Conservative folder
Joe Rogan channel
Mark Dice channel
These new download options only apply to Joe Rogan and Mark Dice. They
don't apply to The Young Turks, which are still using the *previous* set
of download options. They don't apply to CollegeHumor or PewDiePie,
which are still using the *default* download options.
Future plans
------------
- Fix the endless crashes, somehow
- Support for multiple databases (so you can store videos on two
external hard drives at the same time)
- Add download scheduling
- Add video archiving
- Allow selection of multiple videos in the catalogue, so the same
action can be applied to all of them at the same time
- Tie channels and playlists together, so that they won't both download
the same video
- Add tooltips for everything
- Add more youtube-dl options
Known issues
------------
- Tartube crashes continuously and often
- Alphabetic sorting of channels/playlists/folders doesn't always work
as intended, due to an unresolved Gtk issue
- Channels/playlists/folder selection does not always work as intended,
due to an unresolved Gtk issue
- Users can type in comboboxes, but this should not be possible
Contributing
------------
- Report a bug: Use the Github
`issues <https://github.com/axcore/tartube/issues>`__ page
Authors
-------
See the `AUTHORS <AUTHORS>`__ file.
License
-------
Tartube is licensed under the `GNU General Public License
v3.0 <https://www.gnu.org/licenses/gpl-3.0.en.html>`__.
✨🍰✨

2
__init__.py
View File

@ -1 +1 @@
name = "tartube"

1
docs/empty.md Normal file
View File

@ -0,0 +1 @@
#Tartube

8
lib/config.py
View File

@ -31,11 +31,11 @@ import os
# Import our modules
import constants
import __main__
import mainapp
import media
import utils
from . import constants
from . import mainapp
from . import media
from . import utils
# Classes

4
lib/constants.py
View File

@ -49,8 +49,8 @@ FILESIZE_METRIC_LIST = [
# Main stages of the download operation
MAIN_STAGE_QUEUED = 'Queued'
MAIN_STAGE_ACTIVE = 'Active'
MAIN_STAGE_PAUSED = 'Paused'
MAIN_STAGE_COMPLETED = 'Completed'
MAIN_STAGE_PAUSED = 'Paused' # (not actually used)
MAIN_STAGE_COMPLETED = 'Completed' # (not actually used)
MAIN_STAGE_ERROR = 'Error'
# Sub-stages of the 'Active' stage
ACTIVE_STAGE_PRE_PROCESS = 'Pre-processing'

448
lib/downloads.py
View File

@ -31,7 +31,7 @@ import datetime
import json
import signal
import os
import Queue
import queue
import requests
import subprocess
import sys
@ -40,16 +40,11 @@ import time
# Import our modules
from constants import MAIN_STAGE_QUEUED, MAIN_STAGE_ACTIVE, \
MAIN_STAGE_PAUSED, MAIN_STAGE_COMPLETED, MAIN_STAGE_ERROR, \
ACTIVE_STAGE_PRE_PROCESS, ACTIVE_STAGE_DOWNLOAD, ACTIVE_STAGE_POST_PROCESS, \
ACTIVE_STAGE_CHECKING, COMPLETED_STAGE_FINISHED, COMPLETED_STAGE_WARNING, \
COMPLETED_STAGE_ALREADY, ERROR_STAGE_ERROR, ERROR_STAGE_STOPPED, \
ERROR_STAGE_ABORT
import mainapp
import media
import options
import utils
from . import constants
from . import mainapp
from . import media
from . import options
from . import utils
# Decorator to add thread synchronisation to some functions in the
@ -184,8 +179,8 @@ class DownloadManager(threading.Thread):
download_item_obj = self.download_list_obj.fetch_next_item()
# Exit this loop when there are no more downloads.DownloadItem
# objects whose .status is MAIN_STAGE_QUEUED, and when all
# workers have finished their downloads
# objects whose .status is constants.MAIN_STAGE_QUEUED, and when
# all workers have finished their downloads
# Otherwise, wait for an available downloads.DownloadWorker, and
# then assign the next downloads.DownloadItem to it
if not download_item_obj:
@ -210,7 +205,7 @@ class DownloadManager(threading.Thread):
# downloads.DownloadItem
self.download_list_obj.change_item_stage(
download_item_obj.dbid,
MAIN_STAGE_ACTIVE,
constants.MAIN_STAGE_ACTIVE,
)
# Update the main window's progress bar
self.job_count += 1
@ -712,7 +707,7 @@ class DownloadList(object):
dbid (int): The specified item's .dbid
new_stage: The new download stage, one of the values imported from
constants.py (e.g. MAIN_STAGE_QUEUED)
constants.py (e.g. constants.MAIN_STAGE_QUEUED)
"""
@ -816,8 +811,8 @@ class DownloadList(object):
for dbid in self.download_item_list:
this_item = self.download_item_dict[dbid]
# Don't return an item that's marked as MAIN_STAGE_ACTIVE
if this_item.stage == MAIN_STAGE_QUEUED:
# Don't return an item that's marked as constants.MAIN_STAGE_ACTIVE
if this_item.stage == constants.MAIN_STAGE_QUEUED:
return this_item
return None
@ -895,7 +890,7 @@ class DownloadItem(object):
# A unique ID for this object
self.dbid = dbid
# The current download stage
self.stage = MAIN_STAGE_QUEUED
self.stage = constants.MAIN_STAGE_QUEUED
class VideoDownloader(object):
@ -984,8 +979,8 @@ class VideoDownloader(object):
# This object reads from the child process STDOUT and STDERR in an
# asynchronous way
# Standard Python synchronised queue classes
self.stdout_queue = Queue.Queue()
self.stderr_queue = Queue.Queue()
self.stdout_queue = queue.Queue()
self.stderr_queue = queue.Queue()
# The downloads.PipeReader objects created to handle reading from the
# pipes
self.stdout_reader = PipeReader(self.stdout_queue)
@ -1064,7 +1059,7 @@ class VideoDownloader(object):
# Public class methods
def do_download(self):
def OLDdo_download(self):
"""Called by downloads.DownloadWorker.run().
@ -1123,8 +1118,9 @@ class VideoDownloader(object):
# standard format, specified in the comments for
# self.extract_stdout_data()
dl_stat_dict = self.extract_stdout_data(stdout)
# If the job's status is COMPLETED_STAGE_ALREADY or
# ERROR_STAGE_ABORT, set our self.return_code IV
# If the job's status is constants.COMPLETED_STAGE_ALREADY
# or constants.ERROR_STAGE_ABORT, set our
# self.return_code IV
self.extract_stdout_status(dl_stat_dict)
# Pass the dictionary on to self.download_worker_obj so the
# main window can be updated
@ -1183,6 +1179,130 @@ class VideoDownloader(object):
# Pass the result back to the parent downloads.DownloadWorker object
return self.return_code
def do_download(self):
"""Called by downloads.DownloadWorker.run().
Based on YoutubeDLDownloader.download().
Downloads video(s) from a URL described by self.download_item_obj.
Returns:
The final return code, a value in the range 0-5 (as described
above)
"""
# Import the main application (for convenience)
app_obj = self.download_manager_obj.app_obj
# Set the default return code. Everything is OK unless we encounter
# any problems
self.return_code = self.OK
# Reset the errors/warnings stored in the media data object, the last
# time it was checked/downloaded
self.download_item_obj.media_data_obj.reset_error_warning()
# Prepare a system command...
cmd_list = self.get_system_cmd()
# ...and create a new child process using that command
self.create_child_process(cmd_list)
# So that we can read from the child process STDOUT and STDERR, attach
# a file descriptor to the PipeReader objects
if self.child_process is not None:
self.stdout_reader.attach_file_descriptor(
self.child_process.stdout,
)
self.stderr_reader.attach_file_descriptor(
self.child_process.stderr,
)
# While downloading the video, update the callback function with
# the status of the current job
while self.is_child_process_alive():
# Read from the child process STDOUT, and convert into unicode for
# Python's convenience
while not self.stdout_queue.empty():
# # (Convert Python2 to Python3)
# stdout = self.stdout_queue.get_nowait().rstrip()
# stdout = utils.convert_item(stdout, to_unicode=True)
stdout = self.stdout_queue.get_nowait().rstrip().decode('utf-8')
if stdout:
# Convert the statistics into a python dictionary in a
# standard format, specified in the comments for
# self.extract_stdout_data()
dl_stat_dict = self.extract_stdout_data(stdout)
# If the job's status is constants.COMPLETED_STAGE_ALREADY
# or constants.ERROR_STAGE_ABORT, set our
# self.return_code IV
self.extract_stdout_status(dl_stat_dict)
# Pass the dictionary on to self.download_worker_obj so the
# main window can be updated
self.download_worker_obj.data_callback(dl_stat_dict)
if (app_obj.ytdl_write_stdout_flag):
print(stdout)
# The child process has finished
while not self.stderr_queue.empty():
# Read from the child process STDERR queue (we don't need to read
# it in real time), and convert into unicode for python's
# convenience
# # (Convert Python2 to Python3)
# stderr = self.stderr_queue.get_nowait().rstrip()
# stderr = utils.convert_item(stderr, to_unicode=True)
stderr = self.stderr_queue.get_nowait().rstrip().decode('utf-8')
if self.is_warning(stderr):
self.set_return_code(self.WARNING)
self.download_item_obj.media_data_obj.set_warning(stderr)
else:
self.set_return_code(self.ERROR)
self.download_item_obj.media_data_obj.set_error(stderr)
if (app_obj.ytdl_write_stderr_flag):
print(stderr)
# We also set the return code to self.ERROR if the download didn't
# start or if the child process return code is greater than 0
# Original notes from youtube-dl-gui:
# NOTE: In Linux if the called script is just empty Python exits
# normally (ret=0), so we cant detect this or similar cases
# using the code below
# NOTE: In Unix a negative return code (-N) indicates that the child
# was terminated by signal N (e.g. -9 = SIGKILL)
if self.child_process is None:
self.set_return_code(self.ERROR)
self.download_item_obj.media_data_obj.set_error(
'Download did not start',
)
elif self.child_process.returncode > 0:
self.set_return_code(self.ERROR)
self.download_item_obj.media_data_obj.set_error(
'Child process exited with non-zero code: {}'.format(
self.child_process.returncode,
)
)
# Pass a dictionary of values to downloads.DownloadWorker, confirming
# the result of the job. The values are passed on to the main
# window
self.last_data_callback()
# Pass the result back to the parent downloads.DownloadWorker object
return self.return_code
def close(self):
@ -1315,7 +1435,7 @@ class VideoDownloader(object):
)
def confirm_sim_video(self, json_dict):
def OLDconfirm_sim_video(self, json_dict):
"""Called by self.extract_stdout_data().
@ -1515,6 +1635,208 @@ class VideoDownloader(object):
# Update the main window
app_obj.announce_video_download(self.download_item_obj, video_obj)
def confirm_sim_video(self, json_dict):
"""Called by self.extract_stdout_data().
After a successful simulated download, youtube-dl presents us with JSON
data for the video. Use that data to update everything.
Args:
json_dict (dict): JSON data from STDOUT, converted into a python
dictionary
"""
# IMport the main application (for convenience)
app_obj = self.download_manager_obj.app_obj
# From the JSON dictionary, extract the data we need
if '_filename' in json_dict:
full_path = json_dict['_filename']
path, filename, extension = self.extract_filename(full_path)
else:
return app_obj.system_error(
302,
'Missing filename in JSON data',
)
if 'upload_date' in json_dict:
# date_string in form YYYYMMDD
date_string = json_dict['upload_date']
dt_obj = datetime.datetime.strptime(date_string, '%Y%m%d')
upload_time = dt_obj.strftime('%s')
else:
upload_time = None
if 'duration' in json_dict:
duration = json_dict['duration']
else:
duration = None
if 'title' in json_dict:
name = json_dict['title']
else:
name = None
if 'description' in json_dict:
descrip = json_dict['description']
else:
descrip = None
if 'thumbnail' in json_dict:
thumbnail = json_dict['thumbnail']
else:
thumbnail = None
if 'webpage_url' in json_dict:
source = json_dict['webpage_url']
else:
source = None
if 'playlist_index' in json_dict:
playlist_index = json_dict['playlist_index']
else:
playlist_index = None
# Does an existing media.Video object match this video?
media_data_obj = self.download_item_obj.media_data_obj
video_obj = None
if isinstance(media_data_obj, media.Video):
video_obj = media_data_obj
else:
# media_data_obj is a media.Channel or media.Playlist object. Check
# its child objects, looking for a matching video
# (video_obj is set to None, if no match is found)
video_obj = media_data_obj.find_matching_video(app_obj, filename)
if not video_obj:
# No matching media.Video object found, so create a new one
video_obj = app_obj.create_video_from_download(
self.download_item_obj,
path,
filename,
extension,
)
# Update its IVs with the JSON information we extracted
if name is not None:
video_obj.set_name(name)
if upload_time is not None:
video_obj.set_upload_time(upload_time)
if duration is not None:
video_obj.set_duration(duration)
if source is not None:
video_obj.set_source(source)
if descrip is not None:
video_obj.set_video_descrip(
descrip,
app_obj.main_win_obj.long_string_max_len,
)
# Only save the playlist index when this video is actually stored
# inside a media.Playlist object
if isinstance(video_obj.parent_obj, media.Playlist) \
and playlist_index is not None:
video_obj.set_index(playlist_index)
else:
# If the 'Add videos' button was used, the path/filename/extension
# won't be set yet
if not video_obj.file_dir and full_path:
video_obj.set_file(path, filename, extension)
# Update any video object IVs that are not set
if video_obj.name == app_obj.default_video_name \
and name is not None:
video_obj.set_name(name)
if not video_obj.upload_time and upload_time is not None:
video_obj.set_upload_time(upload_time)
if not video_obj.duration and duration is not None:
video_obj.set_duration(duration)
if not video_obj.source and source is not None:
video_obj.set_source(source)
if not video_obj.descrip and descrip is not None:
video_obj.set_video_descrip(
descrip,
app_obj.main_win_obj.long_string_max_len,
)
# Only save the playlist index when this video is actually stored
# inside a media.Playlist object
if not video_obj.index \
and isinstance(video_obj.parent_obj, media.Playlist) \
and playlist_index is not None:
video_obj.set_index(playlist_index)
# Deal with the video description, JSON data and thumbnail, according
# to the settings in options.OptionsManager
options_dict =self.download_worker_obj.options_manager_obj.options_dict
if descrip and options_dict['write_description']:
descrip_path = os.path.join(path, filename + '.description')
if not options_dict['sim_keep_description']:
descrip_path = utils.convert_path_to_temp(
app_obj,
descrip_path,
)
# (Don't replace a file that already exists)
if not os.path.isfile(descrip_path):
# # Convert Python2 to Python3
# fh = open(descrip_path, 'w')
fh = open(descrip_path, 'wb')
fh.write(descrip.encode('utf-8'))
fh.close()
if options_dict['write_info']:
json_path = os.path.join(path, filename + '.info.json')
if not options_dict['sim_keep_info']:
json_path = utils.convert_path_to_temp(app_obj, json_path)
if not os.path.isfile(json_path):
with open(json_path, 'w') as outfile:
json.dump(json_dict, outfile, indent=4)
if thumbnail and options_dict['write_thumbnail']:
# Download the thumbnail, if we don't already have it
# The thumbnail's URL is something like
# 'https://i.ytimg.com/vi/abcdefgh/maxresdefault.jpg'
# When saved to disc by youtube-dl, the file is given the same name
# as the video (but with a different extension)
# Get the thumbnail's extension...
remote_file, remote_ext = os.path.splitext(thumbnail)
# ...and thus get the filename used by youtube-dl when storing the
# thumbnail locally
thumb_path = os.path.join(
video_obj.file_dir,
video_obj.file_name + remote_ext,
)
if not options_dict['sim_keep_thumbnail']:
thumb_path = utils.convert_path_to_temp(app_obj, thumb_path)
if not os.path.isfile(thumb_path):
request_obj = requests.get(thumbnail)
with open(thumb_path, 'wb') as outfile:
outfile.write(request_obj.content)
# Update the main window
app_obj.announce_video_download(self.download_item_obj, video_obj)
def create_child_process(self, cmd_list):
@ -1660,7 +1982,7 @@ class VideoDownloader(object):
stdout_list[0] = stdout_list[0].lstrip('\r')
if stdout_list[0] == '[download]':
dl_stat_dict['status'] = ACTIVE_STAGE_DOWNLOAD
dl_stat_dict['status'] = constants.ACTIVE_STAGE_DOWNLOAD
# Get path, filename and extension
if stdout_list[1] == 'Destination:':
@ -1705,7 +2027,7 @@ class VideoDownloader(object):
# Get file already downloaded status
if stdout_list[-1] == 'downloaded':
dl_stat_dict['status'] = COMPLETED_STAGE_ALREADY
dl_stat_dict['status'] = constants.COMPLETED_STAGE_ALREADY
path, filename, extension = self.extract_filename(
' '.join(stdout_with_spaces_list[1:-4]),
)
@ -1718,14 +2040,14 @@ class VideoDownloader(object):
# Get filesize abort status
if stdout_list[-1] == 'Aborting.':
dl_stat_dict['status'] = ERROR_STAGE_ABORT
dl_stat_dict['status'] = constants.ERROR_STAGE_ABORT
elif stdout_list[0] == '[hlsnative]':
# Get information from the native HLS extractor (see
# https://github.com/rg3/youtube-dl/blob/master/youtube_dl/
# downloader/hls.py#L54
dl_stat_dict['status'] = ACTIVE_STAGE_DOWNLOAD
dl_stat_dict['status'] = constants.ACTIVE_STAGE_DOWNLOAD
if len(stdout_list) == 7:
segment_no = float(stdout_list[6])
@ -1741,7 +2063,7 @@ class VideoDownloader(object):
# A successful video download is announced in one of several ways.
# Use the first announcement to update self.video_check_dict, and
# ignore subsequent announcements
dl_stat_dict['status'] = ACTIVE_STAGE_POST_PROCESS
dl_stat_dict['status'] = constants.ACTIVE_STAGE_POST_PROCESS
# Get the final file extension after the merging process has
# completed
@ -1805,7 +2127,7 @@ class VideoDownloader(object):
self.video_total += 1
dl_stat_dict['playlist_size'] = self.video_total
dl_stat_dict['status'] = ACTIVE_STAGE_CHECKING
dl_stat_dict['status'] = constants.ACTIVE_STAGE_CHECKING
elif stdout_list[0][0] != '[' or stdout_list[0] == '[debug]':
@ -1815,7 +2137,7 @@ class VideoDownloader(object):
else:
# The download has started
dl_stat_dict['status'] = ACTIVE_STAGE_PRE_PROCESS
dl_stat_dict['status'] = constants.ACTIVE_STAGE_PRE_PROCESS
return dl_stat_dict
@ -1827,9 +2149,10 @@ class VideoDownloader(object):
Based on YoutubeDLDownloader._extract_info().
If the job's status is COMPLETED_STAGE_ALREADY or ERROR_STAGE_ABORT,
translate that into a new value for the return code, and then use that
value to actually set self.return_code (which halts the download).
If the job's status is constants.COMPLETED_STAGE_ALREADY or
constants.ERROR_STAGE_ABORT, translate that into a new value for the
return code, and then use that value to actually set self.return_code
(which halts the download).
Args:
@ -1840,11 +2163,11 @@ class VideoDownloader(object):
"""
if 'status' in dl_stat_dict:
if dl_stat_dict['status'] == COMPLETED_STAGE_ALREADY:
if dl_stat_dict['status'] == constants.COMPLETED_STAGE_ALREADY:
self.set_return_code(self.ALREADY)
dl_stat_dict['status'] = None
if dl_stat_dict['status'] == ERROR_STAGE_ABORT:
if dl_stat_dict['status'] == constants.ERROR_STAGE_ABORT:
self.set_return_code(self.FILESIZE_ABORT)
dl_stat_dict['status'] = None
@ -1944,23 +2267,23 @@ class VideoDownloader(object):
dl_stat_dict = {}
if self.return_code == self.OK:
dl_stat_dict['status'] = COMPLETED_STAGE_FINISHED
dl_stat_dict['status'] = constants.COMPLETED_STAGE_FINISHED
elif self.return_code == self.ERROR:
dl_stat_dict['status'] = MAIN_STAGE_ERROR
dl_stat_dict['status'] = constants.MAIN_STAGE_ERROR
dl_stat_dict['eta'] = ''
dl_stat_dict['speed'] = ''
elif self.return_code == self.WARNING:
dl_stat_dict['status'] = COMPLETED_STAGE_WARNING
dl_stat_dict['status'] = constants.COMPLETED_STAGE_WARNING
dl_stat_dict['eta'] = ''
dl_stat_dict['speed'] = ''
elif self.return_code == self.STOPPED:
dl_stat_dict['status'] = ERROR_STAGE_STOPPED
dl_stat_dict['status'] = constants.ERROR_STAGE_STOPPED
dl_stat_dict['eta'] = ''
dl_stat_dict['speed'] = ''
elif self.return_code == self.ALREADY:
dl_stat_dict['status'] = COMPLETED_STAGE_ALREADY
dl_stat_dict['status'] = constants.COMPLETED_STAGE_ALREADY
else:
dl_stat_dict['status'] = ERROR_STAGE_ABORT
dl_stat_dict['status'] = constants.ERROR_STAGE_ABORT
# Use some empty values in dl_stat_dict so that the Progress Tab
# doesn't show arbitrary data from the last file downloaded
@ -2033,7 +2356,7 @@ class PipeReader(threading.Thread):
process pipes in an asynchronous way.
Args:
queue (Queue.Queue): Python queue to store the output of the child
queue (queue.Queue): Python queue to store the output of the child
process.
Warnings:
@ -2075,7 +2398,7 @@ class PipeReader(threading.Thread):
# Public class methods
def run(self):
def OLDOLDrun(self):
"""Called as a result of self.__init__().
@ -2103,6 +2426,42 @@ class PipeReader(threading.Thread):
time.sleep(self.sleep_time)
def run(self):
"""Called as a result of self.__init__().
Reads from STDOUT or STERR using the attached filed descriptor.
"""
# Use this flag so that the loop can ignore FFmpeg error messsages
# (because the parent VideoDownloader object shouldn't use that as a
# serious error)
ignore_line = False
while self.running_flag:
if self.file_descriptor is not None:
# # Convert Python2 to Python3 - the for loop no longer
# # terminates, but produces endless <b''> strings instead
# for line in iter(self.file_descriptor.readline, str('')):
for line in iter(self.file_descriptor.readline, str('')):
if line == b'':
break
# # Convert Python2 to Python3
# if str('ffmpeg version') in line:
if str.encode('ffmpeg version') in line:
ignore_line = True
if not ignore_line:
self.output_queue.put_nowait(line)
self.file_descriptor = None
ignore_line = False
time.sleep(self.sleep_time)
def attach_file_descriptor(self, filedesc):
@ -2135,4 +2494,3 @@ class PipeReader(threading.Thread):
self.running_flag = False
super(PipeReader, self).join(timeout)

44
lib/mainapp.py
View File

@ -51,17 +51,17 @@ except:
# Import our modules
import config
import downloads
import files
import __main__
import mainwin
import media
import options
import refresh
import testing
import updates
import utils
from . import config
from . import downloads
from . import files
from . import mainwin
from . import media
from . import options
from . import refresh
from . import testing
from . import updates
from . import utils
# Classes
@ -415,30 +415,34 @@ class TartubeApp(Gtk.Application):
# Debugging flags (can only be set by editing the source code)
# Delete the config file and the contents of Tartube's data directory
# on startup
self.debug_delete_data_flag = False
# self.debug_delete_data_flag = False
self.debug_delete_data_flag = True
# In the main window's menu, show a menu item for adding a set of
# media data objects for testing
self.debug_test_media_menu_flag = False
# self.debug_test_media_menu_flag = False
self.debug_test_media_menu_flag = True
# In the main window's toolbar, show a toolbar item for adding a set of
# media data objects for testing
self.debug_test_media_toolbar_flag = False
# Show an dialogue window with 'Tartube is already running!' if the
# user tries to open a second instance of Tartube
self.debug_warn_multiple_flag = False
# self.debug_warn_multiple_flag = False
self.debug_warn_multiple_flag = True
# Open the main window in the top-left corner of the desktop
self.debug_open_top_left_flag = False
# self.debug_open_top_left_flag = False
self.debug_open_top_left_flag = True
# Automatically open the system preferences window on startup
self.debug_open_pref_win_flag = False
# For Tartube developers who don't want to manually change
# self.ytdl_path and self.ytdl_update_current on every startup
# (assuming that self.debug_delete_data_flag is True), modify those
# IVs
self.debug_modify_ytdl_flag = False
self.debug_ytdl_path = None
self.debug_ytdl_update_current = None
# self.debug_modify_ytdl_flag = True
# self.debug_ytdl_path = 'youtube-dl'
# self.debug_ytdl_update_current = 'Update using pip'
# self.debug_modify_ytdl_flag = False
# self.debug_ytdl_path = None
# self.debug_ytdl_update_current = None
self.debug_modify_ytdl_flag = True
self.debug_ytdl_path = 'youtube-dl'
self.debug_ytdl_update_current = 'Update using pip'
def do_startup(self):

73
lib/mainwin.py
View File

@ -40,13 +40,14 @@ import time
# Import our modules
from . import config
from . import constants
#from . import __main__
import __main__
import config
import constants
import mainapp
import media
import options
import utils
from . import mainapp
from . import media
from . import options
from . import utils
# Classes
@ -1315,7 +1316,7 @@ class MainWin(Gtk.ApplicationWindow):
return 0
def video_catalogue_auto_sort(self, row1, row2, data, notify):
def OLDvideo_catalogue_auto_sort(self, row1, row2, data, notify):
"""Sorting function created by self.videos_tab.
@ -1359,6 +1360,64 @@ class MainWin(Gtk.ApplicationWindow):
else:
return -1
def video_catalogue_auto_sort(self, row1, row2, data, notify):
"""Sorting function created by self.videos_tab.
Automatically sorts rows in the Video Catalogue.
Args:
row1, row2 (mainwin.CatalogueRow): Two rows in the liststore, one
of which must be sorted before the other
data (None): Ignored
notify (False): Ignored
Returns:
-1 if row1 comes before row2, 1 if row2 comes before row1, 0 if
their order should not be changed
"""
# Get the media.Video objects displayed on each row
obj1 = row1.video_obj
obj2 = row2.video_obj
# Sort videos by playlist index (if set), then by upload time, and then
# by receive (download) time
if obj1.index is not None and obj2.index is not None:
if obj1.index < obj2.index:
return -1
else:
return 1
# # Convert Python2 to Python3
# elif obj1.upload_time < obj2.upload_time:
# return 1
# elif obj1.upload_time == obj2.upload_time:
# if obj1.receive_time < obj2.receive_time:
# return -1
# elif obj1.receive_time == obj2.receive_time:
# return 0
# else:
# return 1
elif obj1.upload_time is not None and obj2.upload_time is not None:
if obj1.upload_time < obj2.upload_time:
return 1
elif obj1.upload_time == obj2.upload_time:
if obj1.receive_time < obj2.receive_time:
return -1
elif obj1.receive_time == obj2.receive_time:
return 0
else:
return 1
else:
return -1
else:
return 0
# (Video Index)

120
lib/media.py
View File

@ -26,14 +26,14 @@
# Import other modules
import datetime
import functools
import os
import time
# Import our modules
import functools
import mainapp
import time
import utils
from . import mainapp
from . import utils
# Classes
@ -401,7 +401,7 @@ class GenericRemoteContainer(GenericContainer):
self.vid_count += 1
def do_sort(self, obj1, obj2):
def OLDdo_sort(self, obj1, obj2):
"""Sorting function used by functools.cmp_to_key(), and called by
self.sort_children().
@ -435,6 +435,45 @@ class GenericRemoteContainer(GenericContainer):
else:
return -1
def do_sort(self, obj1, obj2):
"""Sorting function used by functools.cmp_to_key(), and called by
self.sort_children().
Sort videos by upload time, with the most recent video first.
When downloading a channel or playlist, we assume that YouTube (etc)
supplies us with the most recent upload first. Therefore, when the
upload time is the same, sort by the order in youtube-dl fetches the
videos.
Args:
obj1, obj2 (media.Video) - Video objects being sorted
Returns:
-1 if obj1 comes first, 1 if obj2 comes first, 0 if they are equal
"""
# # Convert Python2 to Python3
# if obj1.upload_time < obj2.upload_time:
# return 1
if obj1.upload_time is None or obj2.upload_time is None:
return 0
elif obj1.upload_time < obj2.upload_time:
return 1
elif obj1.upload_time == obj2.upload_time:
if obj1.receive_time < obj2.receive_time:
return -1
elif obj1.receive_time == obj2.receive_time:
return 0
else:
return 1
else:
return -1
def find_matching_video(self, app_obj, name):
@ -1287,7 +1326,7 @@ class Folder(GenericContainer):
# def del_child(): # Inherited from GenericContainer
def do_sort(self, obj1, obj2):
def OLDdo_sort(self, obj1, obj2):
"""Sorting function used by functools.cmp_to_key(), and called by
self.sort_children().
@ -1351,6 +1390,75 @@ class Folder(GenericContainer):
else:
return 0
def do_sort(self, obj1, obj2):
"""Sorting function used by functools.cmp_to_key(), and called by
self.sort_children().
Sorts the child media.Video, media.Channel, media.Playlist and
media.Folder objects.
Firstly, sort by class - folders, channels/playlists, then videos.
Within folders, channels and playlists, sort alphabetically. Within
videos, sort by upload time.
Args:
obj1, obj2 (media.Video, media.Channel, media.Playlist or
media.Folder) - Media data objects being sorted
Returns:
-1 if obj1 comes first, 1 if obj2 comes first, 0 if they are equal
"""
if str(obj1.__class__) == str(obj2.__class__) \
or (
isinstance(obj1, GenericRemoteContainer) \
and isinstance(obj2, GenericRemoteContainer)
):
if isinstance(obj1, Video):
# # Convert Python2 to Python3
# if obj1.upload_time < obj2.upload_time:
# return 1
if obj1.upload_time is None or obj2.upload_time is None:
return 0
elif obj1.upload_time < obj2.upload_time:
return 1
elif obj1.upload_time == obj2.upload_time:
if obj1.receive_time < obj2.receive_time:
return -1
elif obj1.receive_time == obj2.receive_time:
return 0
else:
return 1
else:
return -1
else:
if obj1.name.lower() < obj2.name.lower():
return -1
elif obj1.name.lower() == obj2.name.lower():
return 0
else:
return 1
else:
if isinstance(obj1, Folder):
return -1
elif isinstance(obj2, Folder):
return 1
elif isinstance(obj1, Channel) or isinstance(obj1, Playlist):
return -1
elif isinstance(obj2, Channel) or isinstance(obj2, Playlist):
return 1
else:
return 0
# def find_child_index(): # Inherited from GenericContainer

8
lib/options.py
View File

@ -29,10 +29,10 @@ import os
# Import our modules
import constants
import mainapp
import media
import utils
from . import constants
from . import mainapp
from . import media
from . import utils
# Classes

4
lib/refresh.py
View File

@ -25,13 +25,13 @@
# Import other modules
import constants
import os
import threading
# Import our modules
import media
from . import constants
from . import media
# Classes

102
lib/updates.py
View File

@ -26,7 +26,8 @@
# Import other modules
import os
import Queue
import queue
import re
import requests
import subprocess
import sys
@ -34,9 +35,8 @@ import threading
# Import our modules
import downloads
import re
import utils
from . import downloads
from . import utils
# Classes
@ -74,8 +74,8 @@ class UpdateManager(threading.Thread):
# This object reads from the child process STDOUT and STDERR in an
# asynchronous way
# Standard Python synchronised queue classes
self.stdout_queue = Queue.Queue()
self.stderr_queue = Queue.Queue()
self.stdout_queue = queue.Queue()
self.stderr_queue = queue.Queue()
# The downloads.PipeReader objects created to handle reading from the
# pipes
self.stdout_reader = downloads.PipeReader(self.stdout_queue)
@ -105,7 +105,7 @@ class UpdateManager(threading.Thread):
# Public class methods
def run(self):
def OLDrun(self):
"""Called as a result of self.__init__().
@ -189,6 +189,94 @@ class UpdateManager(threading.Thread):
else:
self.app_obj.update_manager_finished(True)
def run(self):
"""Called as a result of self.__init__().
Based on code from downloads.VideoDownloader.do_download().
Creates a child process to run the youtube-dl update.
Reads from the child process STDOUT and STDERR, and calls the main
application with the result of the update (success or failure).
"""
# Prepare the system command
# The user can change the system command for updating youtube-dl,
# depending on how it was installed
# (For example, if youtube-dl was installed via pip, then it must be
# updated via pip)
cmd_list \
= self.app_obj.ytdl_update_dict[self.app_obj.ytdl_update_current]
# Create a new child process using that command
self.create_child_process(cmd_list)
# So that we can read from the child process STDOUT and STDERR, attach
# a file descriptor to the PipeReader objects
if self.child_process is not None:
self.stdout_reader.attach_file_descriptor(
self.child_process.stdout,
)
self.stderr_reader.attach_file_descriptor(
self.child_process.stderr,
)
while self.is_child_process_alive():
# Read from the child process STDOUT, and convert into unicode for
# Python's convenience
while not self.stdout_queue.empty():
# # (Convert Python2 to Python3)
# stdout = self.stdout_queue.get_nowait().rstrip()
# stdout = utils.convert_item(stdout, to_unicode=True)
stdout = self.stdout_queue.get_nowait().rstrip().decode('utf-8')
if stdout:
# "It looks like you installed youtube-dl with a package
# manager, pip, setup.py or a tarball. Please use that to
# update."
if re.search('It looks like you installed', stdout):
self.stderr_list.append(stdout)
else:
self.stdout_list.append(stdout)
# The child process has finished
while not self.stderr_queue.empty():
# Read from the child process STDERR queue (we don't need to read
# it in real time), and convert into unicode for python's
# convenience
# # (Convert Python2 to Python3)
# stderr = self.stderr_queue.get_nowait().rstrip()
# stderr = utils.convert_item(stderr, to_unicode=True)
stderr = self.stderr_queue.get_nowait().rstrip().decode('utf-8')
if stderr:
self.stderr_list.append(stderr)
# (Generate our own error messages for debugging purposes, in certain
# situations)
if self.child_process is None:
self.stderr_list.append('Download did not start')
elif self.child_process.returncode > 0:
self.stderr_list.append(
'Child process exited with non-zero code: {}'.format(
self.child_process.returncode,
)
)
# Operation complete; inform the main application of success or failure
if self.stderr_list:
self.app_obj.update_manager_finished(False)
else:
self.app_obj.update_manager_finished(True)
def create_child_process(self, cmd_list):

6
lib/utils.py
View File

@ -36,10 +36,10 @@ import textwrap
# Import our modules
import constants
import mainapp
from . import constants
from . import mainapp
if mainapp.HAVE_VALIDATORS_FLAG:
import validators
from . import validators
# Functions

25
setup.py
View File

@ -21,27 +21,30 @@
# Import modules
from setuptools import setup, find_packages
import setuptools
# Import documents
with open('README.rst') as f:
readme = f.read()
with open('LICENSE') as f:
license = f.read()
#with open('README.rst', 'r') as f:
# long_description = f.read()
#
#with open('LICENSE') as f:
# license = f.read()
# Setup
setup(
setuptools.setup(
name='tartube',
version='0.1.000',
version='0.1.007',
description='GUI front-end for youtube-dl',
long_description=readme,
# long_description=long_description,
long_description="""Tartube is a GUI front-end for youtube-dl, partly based on youtube-dl-gui and written in Python 3 / Gtk 3""",
long_description_content_type='text/markdown',
author='A S Lewis',
author_email='aslewis@cpan.org',
url='https://github.com/axcore/tartube',
license=license,
packages=find_packages(exclude=('tests', 'docs'))
# license=license,
license="""GPL3+""",
packages=setuptools.find_packages()
)

4
tartube.py
View File

@ -34,8 +34,8 @@ from lib import mainapp
# 'Global' variables
__packagename__ = 'tartube'
__version__ = '0.1.000'
__date__ = '27 May 2019'
__version__ = '0.1.007'
__date__ = '28 May 2019'
__copyright__ = 'Copyright \xc2\xa9 2019 A S Lewis'
__license__ = """
Copyright \xc2\xa9 2019 A S Lewis.

1
tests/__init__.py Normal file
View File

@ -0,0 +1 @@