Update to v2.2.0
This commit is contained in:
A S Lewis
parent
3eab9caccf
commit
c8aaf7c024
112
CHANGES
112
CHANGES
@ -1,3 +1,115 @@
|
||||
v2.2.0 (30 Sep 2020)
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
MAJOR NEW FEATURES
|
||||
- Tartube is now confirmed to work on MacOS. See the README file for
|
||||
installation instructions
|
||||
- Tartube can now handle video thumbnails in the .webp format, as long as
|
||||
FFmpeg is installed. Thumbnails are automatically converted to .jpg (either
|
||||
by youtube-dl, or by Tartube itself, as appropriate). This only affects new
|
||||
downloads. If you want to convert .webp thumbnails you've already
|
||||
downloaded, click Operations > Tidy up files..., select 'Convert .webp
|
||||
thumbnails to .jpg using Ffmpeg', and click OK. (This procedure may take a
|
||||
while if there are thousands of thumbnails to convert) (Git #155 and
|
||||
others)
|
||||
- Thumbnails, video description, metadata and annotation files can now be
|
||||
downloaded into a sub-directory, rather than being stored in the same
|
||||
directory as their videos. Thumbnails are stored in /.thumbs, and the
|
||||
others are stored in /.data. On Linux/BSD, those sub-directories are
|
||||
normally invisible by default (typically, pressing CTRL+H will reveal
|
||||
them). This new feature is disabled by default. To enable it, click Edit >
|
||||
Download options... > Files > Write/move files, and select one or more of
|
||||
the checkboxes. The new feature only affects new downloads. If you want to
|
||||
move files you've already downloaded, click Operations > Tidy up files...,
|
||||
select 'Move thumbnails into own folder' and/or 'Move other metadata files
|
||||
into own folder', and click OK (Git #139)
|
||||
- You can now select one or more videos, and process them with FFmpeg directly
|
||||
(in other words, after downloads have finished, and without involving
|
||||
youtube-dl). This will be useful if you want to convert one video format to
|
||||
another, change the frame rate, or with countless other tasks. Just select
|
||||
the video(s), right-click them and select 'Process with FFmpeg...'. Since
|
||||
many FFmpeg procedures require a different output filename, you can specify
|
||||
that, too. Note that FFmpeg sometimes takes a very long time; you should
|
||||
test a procedure with a single video, before trying to process hundreds of
|
||||
them (Git #153)
|
||||
- Tartube can now use forks of youtube-dl, such as youtube-dlc. (Tartube
|
||||
assumes that a fork is still very similar to the original). A fork can be
|
||||
specified in Edit > System preferences... > youtube-dl (Git #158)
|
||||
- New installations of Tartube will now auto-detect the location of youtube-dl,
|
||||
if it is already installed on your system. This will benefit users of the
|
||||
.DEB and .RPM packages, who until now were expected to know how to set the
|
||||
youtube-dl path manually (Git #152)
|
||||
- Ffmpeg and AVConv will now also be auto-detected. If you have installed them
|
||||
in unusual locations, you should specify those locations in Edit >
|
||||
System preferences... > youtube-dl > Ffmpeg / AVConv. If not, there is no
|
||||
need to specify either location; just leave the boxes empty. (None of this
|
||||
applies to MS Windows users)
|
||||
- When Tartube shuts down unexpectedly, it doesn't have time to mark the
|
||||
database as no longer being in use (i.e. doesn't remove the lockfile). The
|
||||
next time Tartube runs, users were prompted to remove the lockfile, then
|
||||
restart Tartube. Many users were unhappy with this situation, so it has
|
||||
been improved. You will still be prompted to remove the lockfile, but there
|
||||
is no longer any need to restart Tartube
|
||||
|
||||
MINOR NEW FEATURES
|
||||
- FFmpeg is now required for a lot of Tartube functionality. On new
|
||||
installations, users who have not yet installed FFmpeg will see some
|
||||
additional nag-boxes, and various hints in other configuration windows
|
||||
(Git #155)
|
||||
- If users downloadd a video, but only its audio, the video appeared in
|
||||
Tartube's database as downloaded. However, when the user clicked on the
|
||||
'Player' label, the audio file was not opened in the system's media player.
|
||||
Tartube now checks for an audio file (as well as video files in different
|
||||
formats), if the video file it was expecting does not exist
|
||||
- When the user clicks Operations > Update youtube-dl, Tartube now
|
||||
automatically makes the Output tab visible. Hopefully this will avoid
|
||||
confusion for new users, who do not notice that the Check all/Download all
|
||||
buttons have been greyed out. This behaviour can be disabled, if required:
|
||||
click Edit > System preferences... > Output > Output Tab, and deselect
|
||||
'During an update operation, automatically switch to the Output Tab'
|
||||
(Git #149)
|
||||
- The invidio.us website has closed. There are many mirrors available. Tartube
|
||||
now uses invidious.site as its default mirror. To specify a different
|
||||
mirror, click Edit > System preferences... > Operations > Downloads
|
||||
- YouTube phased out video annotations in 2019. The Tartube code was unable to
|
||||
download annotation files during a simulated download (for example, with
|
||||
the 'Check all' button). Now that there is no way of testing any fix, the
|
||||
feature has been removed entirely
|
||||
- You can now add automatic custom downloads on a schedule (normal downloads
|
||||
were already available). Click Edit > System preferences... > Scheduling
|
||||
(Git #154)
|
||||
- Reduced the compulsory delay at the end of many types of operation. The
|
||||
delay time is not the same for all types of operation
|
||||
- Tartube debug messages are visible in the terminal window (if open). Before,
|
||||
the user had to edit the source code to enable debug messages. They can
|
||||
now be enabled from within Tartube itself: click Edit > System
|
||||
preferences... > General > Debugging. These settings are not saved, so when
|
||||
Tartube restarts, you will have to re-enable debug messages again
|
||||
- In the Progress and Classic Mode tabs, the name of the incoming file (and
|
||||
other similar columns) are no longer artificially shortened (Git #161)
|
||||
|
||||
MAJOR FIXES
|
||||
- When one channel downloads into videos into another channel's directory,
|
||||
and the other channel is then deleted, the Tartube database did not update
|
||||
itself properly. Fixed
|
||||
- Fixed some issues when using the refresh operation to import videos, that had
|
||||
been downloaded by youtube-dl (without Tartube's) help, into Tartube's
|
||||
database (Git #142)
|
||||
- Tartube can now download videos using the youtube-dl archive file, even when
|
||||
missing video detection is turned on (Git #154)
|
||||
- Tartube sometimes froze on shutdown, after youtube-dl had been updated.
|
||||
Applied the existing MS Windows fix to all operating systems
|
||||
|
||||
MINOR FIXES
|
||||
- Fixed issues when importing a JSON export file on MS Windows, and a different
|
||||
issue error when importing it on Linux
|
||||
- We were not able to fix export issues reported in Git #143, but we have
|
||||
updated the dialogue window, which should give more information about what
|
||||
is causing the error
|
||||
- System folders cannot be deleted. The 'Delete folder' popup menu item is now
|
||||
greyed out. (Nothing happened, even when it was clickable)
|
||||
- Warnings about broken Gtk have been removed (probably permanently)
|
||||
|
||||
v2.1.070 (8 Aug 2020)
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
|
||||
199
README.rst
199
README.rst
@ -41,24 +41,28 @@ Problems can be reported at `our GitHub page <https://github.com/axcore/tartube/
|
||||
- Certain popular websites manipulate search results, repeatedly unsubscribe people from their favourite channels and/or deliberately conceal videos that they don't like. **Tartube** won't do any of those things
|
||||
- **Tartube** is free and open-source software
|
||||
|
||||
2.1 What's new in version 2.1.0
|
||||
2.1 What's new in version 2.2.0
|
||||
-------------------------------
|
||||
|
||||
- For everyone who wants a simpler way to download videos, a new Classic Mode, emulating the look and feel of `youtube-dl-gui <https://mrs0m30n3.github.io/youtube-dl-gui/>`__ - see `6.21 Classic Mode`_
|
||||
- **Tartube** can now detect livestreams, and alert you when they start - see `6.22 Livestreams`_. This feature is EXPERIMENTAL, has only been tested on **YouTube**, and may not be reliable.
|
||||
- If you can contribute a translation to this project, `please read this <docs/translate.rst>`__. As a proof of concept, **Tartube** can now be used with either British or American English
|
||||
- **Tartube** is confirmed to work on MacOS see `5.2 Installation - MacOS`_
|
||||
- For a while, **Tartube** has been unable to display video thumbnails, after **YouTube** started using a format that no-one supports. Both **youtube-dl** and **Tartube** have been updated with workarounds; they will only work if you have installed `FFmpeg <https://ffmpeg.org/>`__ (for help with that, see below)
|
||||
- If you like tidy directories (folders), you can store the thumbnail and metadata files in a sub-folder, leaving the main folder containing only videos. To enable this, click **Edit > Download options... > Files > Write/move files**
|
||||
- Videos can now be sent to FFmpeg directly for processing (in other words, after the download has finished). Right-click one or more videos and select **Process with FFmpeg...**
|
||||
- **Tartube** now supports forks of **youtube-dl**, such as `youtube-dlc <https://github.com/blackjack4494/youtube-dlc>`__
|
||||
|
||||
For a full list of new features and fixes, see `recent changes <CHANGES>`__.
|
||||
|
||||
3 Downloads
|
||||
===========
|
||||
|
||||
Latest version: **v2.1.070 (8 Aug 2020)** (see `recent changes <CHANGES>`__)
|
||||
Latest version: **v2.2.0 (30 Sep 2020)**
|
||||
|
||||
Official packages (also available from the `Github release page <https://github.com/axcore/tartube/releases>`__):
|
||||
|
||||
- `MS Windows (64-bit) installer <https://sourceforge.net/projects/tartube/files/v2.1.070/install-tartube-2.1.070-64bit.exe/download>`__ and `portable edition <https://sourceforge.net/projects/tartube/files/v2.1.070/tartube-portable-64bit.zip/download>`__ from Sourceforge
|
||||
- `MS Windows (32-bit) installer <https://sourceforge.net/projects/tartube/files/v2.1.070/install-tartube-2.1.070-32bit.exe/download>`__ and `portable edition <https://sourceforge.net/projects/tartube/files/v2.1.070/tartube-portable-32bit.zip/download>`__ from Sourceforge
|
||||
- `DEB package (for Debian-based distros, e.g. Ubuntu, Linux Mint) <https://sourceforge.net/projects/tartube/files/v2.1.070/python3-tartube_2.1.070.deb/download>`__ from Sourceforge
|
||||
- `RPM package (for RHEL-based distros, e.g. Fedora) <https://sourceforge.net/projects/tartube/files/v2.1.070/tartube-2.1.070.rpm/download>`__ from Sourceforge
|
||||
- `MS Windows (64-bit) installer <https://sourceforge.net/projects/tartube/files/v2.2.0/install-tartube-2.2.0-64bit.exe/download>`__ and `portable edition <https://sourceforge.net/projects/tartube/files/v2.2.0/tartube-portable-64bit.zip/download>`__ from Sourceforge
|
||||
- `MS Windows (32-bit) installer <https://sourceforge.net/projects/tartube/files/v2.2.0/install-tartube-2.2.0-32bit.exe/download>`__ and `portable edition <https://sourceforge.net/projects/tartube/files/v2.2.0/tartube-portable-32bit.zip/download>`__ from Sourceforge
|
||||
- `DEB package (for Debian-based distros, e.g. Ubuntu, Linux Mint) <https://sourceforge.net/projects/tartube/files/v2.2.0/python3-tartube_2.2.0.deb/download>`__ from Sourceforge
|
||||
- `RPM package (for RHEL-based distros, e.g. Fedora) <https://sourceforge.net/projects/tartube/files/v2.2.0/tartube-2.2.0.rpm/download>`__ from Sourceforge
|
||||
|
||||
There are also some DEB/RPM packages marked STRICT. In these packages, updates to **youtube-dl** from within **Tartube** have been disabled. If **Tartube** is uploaded to a repository with lots of rules, such as the official Debian repository, then you should probably use the STRICT packages.
|
||||
|
||||
@ -69,7 +73,7 @@ Semi-official packages:
|
||||
|
||||
Source code:
|
||||
|
||||
- `Source code <https://sourceforge.net/projects/tartube/files/v2.1.070/tartube_v2.1.070.tar.gz/download>`__ from Sourceforge
|
||||
- `Source code <https://sourceforge.net/projects/tartube/files/v2.2.0/tartube_v2.2.0.tar.gz/download>`__ from Sourceforge
|
||||
- `Source code <https://github.com/axcore/tartube>`__ and `support <https://github.com/axcore/tartube/issues>`__ from GitHub
|
||||
|
||||
4 Quick start guide
|
||||
@ -82,7 +86,7 @@ Source code:
|
||||
- Start **Tartube** from the Start menu, or by clicking the icon on the desktop
|
||||
- When prompted, choose a folder where **Tartube** can store videos
|
||||
- When prompted, let **Tartube** install **youtube-dl** for you
|
||||
- It's strongly recommended that you install **FFmpeg**. From the menu, click **Operations > Install FFmpeg**
|
||||
- It's strongly recommended that you install `FFmpeg <https://ffmpeg.org/>`__ . From the menu, click **Operations > Install FFmpeg**
|
||||
|
||||
If you don't want **Tartube** to add videos to its database, click the **Classic Mode** Tab. If you *do* want to update the database, do this instead:
|
||||
|
||||
@ -98,7 +102,7 @@ If you don't want **Tartube** to add videos to its database, click the **Classic
|
||||
-------------------
|
||||
|
||||
- Install **Tartube**, using any of the methods described below
|
||||
- It's strongly recommended that you install `Ffmpeg <https://ffmpeg.org/>`__ or `AVConv <https://sourceforge.io/projects/avconv/>`__, too
|
||||
- It's strongly recommended that you install `Ffmpeg <https://ffmpeg.org/>`__, too
|
||||
- Run **Tartube**
|
||||
- When prompted, choose a directory where **Tartube** can store videos
|
||||
- Install **youtube-dl** by clicking **Operations > Update youtube-dl**
|
||||
@ -123,7 +127,7 @@ MS Windows users should use the installer `available at the Tartube website <htt
|
||||
|
||||
There is also a portable edition; use this if you want to install **Tartube** onto removable media, such as a USB drive. Download the ZIP file, extract it, and run the file **tartube_portable_64bit.bat** or **tartube_portable_32bit.bat**.
|
||||
|
||||
If you want to use **FFmpeg**, see `6.4 Setting the location of FFmpeg / AVConv`_.
|
||||
It's strongly recommended that you install `Ffmpeg <https://ffmpeg.org/>`__ - see `6.4 Installing FFmpeg / AVConv`_.
|
||||
|
||||
Both the installer and the portable edition include a copy of `AtomicParsley <https://bitbucket.org/jonhedgerows/atomicparsley/wiki/Home>`__, so there is no need to install it yourself.
|
||||
|
||||
@ -202,7 +206,7 @@ MacOS users should use the following procedure (with thanks to JeremyShih):
|
||||
|
||||
**brew install adwaita-icon-theme**
|
||||
|
||||
- It's recommended that you install `Ffmpeg <https://ffmpeg.org/>`__, too
|
||||
- It is strongly recommended that you install `Ffmpeg <https://ffmpeg.org/>`__, too
|
||||
|
||||
**brew install ffmpeg**
|
||||
|
||||
@ -228,6 +232,10 @@ Linux distributions based on Debian, such as Ubuntu and Linux Mint, can install
|
||||
2. **Tartube** asks you to choose a data directory, so do that
|
||||
3. Click **Operations > Update youtube-dl**
|
||||
|
||||
It is strongly recommended that you install `Ffmpeg <https://ffmpeg.org/>`__, too. On most Debian-based systems, you can open a terminal window and run this command:
|
||||
|
||||
**sudo apt-get install ffmpeg**
|
||||
|
||||
5.3.2 Install using the RPM package
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
@ -250,6 +258,14 @@ On Fedora, the procedure is:
|
||||
3. Type: ``pip3 install youtube-dl``
|
||||
4. You can now run **Tartube**.
|
||||
|
||||
It is strongly recommended that you install `Ffmpeg <https://ffmpeg.org/>`__, too. On most RHEL-based systems (for example, Fedora 29-32), you can open a terminal window and run these commands:
|
||||
|
||||
**sudo dnf -y install https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm**
|
||||
|
||||
**sudo dnf -y install https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm**
|
||||
|
||||
**sudo apt-get install ffmpeg**
|
||||
|
||||
5.3.3 Install using the AUR package
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
@ -261,12 +277,16 @@ On Arch-based systems. such as Manjaro, Tartube can be installed using the semi-
|
||||
4. Type: ``makepkg -si``
|
||||
5. You can now run **Tartube**.
|
||||
|
||||
5.3.4 Install using the ebuild/AUR packages
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
It is strongly recommended that you install `Ffmpeg <https://ffmpeg.org/>`__, too. On most Arch-based systems, you can open a terminal window and run this command:
|
||||
|
||||
**sudo pacman -S ffmpeg**
|
||||
|
||||
5.3.4 Install using the ebuild package
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
On Gentoo-based systems, **Tartube** can be installed using the semi-official ebuild package, using the link above.
|
||||
|
||||
Tartube requires `youtube-dl <https://youtube-dl.org/>`__.
|
||||
Tartube requires `youtube-dl <https://youtube-dl.org/>`__. It is strongly recommended that you install `Ffmpeg <https://ffmpeg.org/>`__, too.
|
||||
|
||||
If you're not sure how to install using ebuild, then it might be easier to install from PyPI.
|
||||
|
||||
@ -301,7 +321,7 @@ Here is the procedure for Debian-based distributions, like Ubuntu and Linux Mint
|
||||
5.3.8 Manual installation
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
For any other method of installation, the following dependencies are required:
|
||||
For any other method of installation on Linux/BSD, the following dependencies are required:
|
||||
|
||||
- `Python 3 <https://www.python.org/downloads>`__
|
||||
- `Gtk 3 <https://python-gtk-3-tutorial.readthedocs.io/en/latest/>`__
|
||||
@ -314,7 +334,7 @@ These dependencies are optional, but recommended:
|
||||
- `Python feedparser module <https://pypi.org/project/feedparser/>`__ - enables **Tartube** to detect livestreams
|
||||
- `Python moviepy module <https://pypi.org/project/moviepy/>`__ - if the website doesn't tell **Tartube** about the length of its videos, moviepy can work it out
|
||||
- `Python playsound module <https://pypi.org/project/playsound/>`__ - enables **Tartube** to play an alarm when a livestream starts
|
||||
- `Ffmpeg <https://ffmpeg.org/>`__ or `AVConv <https://sourceforge.io/projects/avconv/>`__ - required for various video post-processing tasks; see the section below if you want to use FFmpeg or AVConv
|
||||
- `Ffmpeg <https://ffmpeg.org/>`__ - required for various video post-processing tasks; see the section below if you want to use FFmpeg
|
||||
- `AtomicParsley <https://bitbucket.org/wez/atomicparsley/src/default/>`__ - required for embedding thumbnails in audio files
|
||||
|
||||
5.3.9 Install from source
|
||||
@ -342,7 +362,7 @@ After installing dependencies (see above):
|
||||
* `6.1 Choose where to save videos`_
|
||||
* `6.2 Check youtube-dl is updated`_
|
||||
* `6.3 Setting youtube-dl's location`_
|
||||
* `6.4 Setting the location of FFmpeg / AVConv`_
|
||||
* `6.4 Installing FFmpeg / AVConv`_
|
||||
* `6.4.1 On MS Windows`_
|
||||
* `6.4.2 On Linux/BSD/MacOS`_
|
||||
* `6.5 Introducing system folders`_
|
||||
@ -380,6 +400,13 @@ After installing dependencies (see above):
|
||||
* `6.22.3 Livestream notifications`_
|
||||
* `6.22.4 Compatible websites`_
|
||||
* `6.23 Detecting missing videos`_
|
||||
* `6.24 More information about FFmpeg and AVConv`_
|
||||
* `6.24.1 Using FFmpeg / AVConv with youtube-dl`_
|
||||
* `6.24.2 Using FFmpeg directly`_
|
||||
* `6.24.3 Changing the filename`_
|
||||
* `6.24.4 Changing the video format`_
|
||||
* `6.24.5 FFmpeg command-line options`_
|
||||
* `6.25 Using youtube-dl forks`_
|
||||
|
||||
6.1 Choose where to save videos
|
||||
-------------------------------
|
||||
@ -429,10 +456,21 @@ On other systems, users can modify **Tartube**'s settings. There are several loc
|
||||
- Try changing the setting **Shell command for update operations**
|
||||
- Try the update operation again
|
||||
|
||||
6.4 Setting the location of FFmpeg / AVConv
|
||||
-------------------------------------------
|
||||
6.4 Installing FFmpeg / AVConv
|
||||
------------------------------
|
||||
|
||||
**youtube-dl** can use the `FFmpeg library <https://ffmpeg.org/>`__ or the `AVConv library <https://sourceforge.io/projects/avconv/>`__ for various video-processing tasks, such as converting video files to audio, and for handling large resolutions (1080p and higher). If you want to use FFmpeg or AVConv, you should first install them on your system.
|
||||
`FFmpeg <https://ffmpeg.org/>`__ and `AVConv <https://sourceforge.io/projects/avconv/>`__ are commonly use for various video-processing tasks.
|
||||
|
||||
**It is strongly recommended that all users install FFmpeg**. Without it, Tartube won't be able to do any of these things:
|
||||
|
||||
- Display thumbnails from **YouTube**
|
||||
- Download high-resolution videos from any website
|
||||
- Download certain other video formats
|
||||
- Convert video files to audio
|
||||
|
||||
**youtube-dl** uses FFmpeg by default, but it can use AVConv for certain tasks.
|
||||
|
||||
For more information about **Tartube**'s use of Ffmpeg and AVConv, see `6.24 More information about FFmpeg and AVConv`_.
|
||||
|
||||
6.4.1 On MS Windows
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
@ -444,13 +482,17 @@ There is no known method of installing a compatible version of AVConv.
|
||||
6.4.2 On Linux/BSD/MacOS
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
On all other operating systems, **youtube-dl** might be able to detect FFmpeg/AVConv without any help from you. If not, you can tell **Tartube** where to find FFmpeg/AVConv in this same tab.
|
||||
On all other operating systems, **Tartube** and **youtube-dl** should be able to use FFmpeg (and AVConv, if it is also installed) without any help from you.
|
||||
|
||||
If the FFmpeg / AVConv executables have been installed to an unusual location, you can tell **Tartube** where to find them.
|
||||
|
||||
.. image:: screenshots/example5.png
|
||||
:alt: Updating ffmpeg
|
||||
:alt: Updating FFmpeg and AVConv
|
||||
|
||||
- Click **Edit > System preferences... > youtube-dl > Preferences**
|
||||
- Click the **Set** button and select the FFmpeg/AVConv executable
|
||||
- Click **Edit > System preferences... > youtube-dl > FFmpeg / AVConv**
|
||||
- Click the **Set** buttons and select the FFmpeg or AVConv executable
|
||||
- Click the **Reset** buttons to remove that selection
|
||||
- Click the the **Use default path** buttons to explictly use the normal location for the executables
|
||||
|
||||
6.5 Introducing system folders
|
||||
------------------------------
|
||||
@ -549,7 +591,7 @@ Once you've finished adding videos, channels, playlists and folders, you can mak
|
||||
- **Download** - Actually downloads the videos. If you have disabled downloads for a particular item, **Tartube** will just fetch information about it instead
|
||||
- **Custom download** - Downloads videos in a non-standard way; see `6.13 Custom downloads`_
|
||||
- **Refresh** - Examines your filesystem. If you have manually copied any videos into **Tartube**'s data directory, those videos are added to **Tartube**'s database
|
||||
- **Update** - Installs or updates **youtube-dl**, as described in `6.2 Check youtube-dl is updated`_. Also installs FFmpeg (on MS Windows only); see `6.4 Setting the location of FFmpeg / AVConv`_
|
||||
- **Update** - Installs or updates **youtube-dl**, as described in `6.2 Check youtube-dl is updated`_. Also installs FFmpeg (on MS Windows only); see `6.4 Installing FFmpeg / AVConv`_
|
||||
- **Info** - Fetches information about a particular video: either the available video/audio formats, or the available subtitles
|
||||
- **Tidy** - Tidies up **Tartube**'s data directory, as well as checking that downloaded videos still exist and are not corrupted
|
||||
|
||||
@ -643,7 +685,7 @@ If **Tartube** can't download a video from YouTube, it's sometimes possible to o
|
||||
|
||||
This only works when requesting individual videos, not whole channels or playlists. You should normally enable independent downloads as well (as described above)
|
||||
|
||||
There are a number of alternative YouTube front-ends available. `HookTube <https://hooktube.com/>`__ and `Invidious <https://invidio.us/>`__ are, at the time of writing, the most famous. However, you can specify any alternative website you like.
|
||||
There are a number of alternative YouTube front-ends available, besides `HookTube <https://hooktube.com/>`__. The original `Invidious <https://invidio.us/>`__ closed in September 2020, but there are a number of mirrors, such as `this one <https://invidious.site/>`__. To get a list of mirrors, `see this page <https://instances.invidio.us/>`__, or use your favourite search engine.
|
||||
|
||||
When specifying an alternative website, it's very important that you type the *exact text* that replaces **youtube.com** in a video's URL. For example, you must type **hooktube.com** not **www.hooktube.com** or **http://www.hooktube.com/**.
|
||||
|
||||
@ -670,7 +712,13 @@ If you've downloaded a video, you can watch it by clicking the word **Player**.
|
||||
|
||||
If you haven't downloaded the video yet, you can watch it online by clicking the word **Website** or **YouTube**. (One or the other will be visible).
|
||||
|
||||
If it's a YouTube video that is restricted (not available in certain regions, or without confirming your age), it's sometimes possible to watch the same video without restrictions on alternative website, such as `HookTube <https://hooktube.com/>`__ or `Invidious <https://invidio.us/>`__.
|
||||
Restricted YouTube videos (not available in your region, or not visible without a Google account) can usually be watched without restrictions on an alternative website, such as `HookTube <https://hooktube.com/>`__ or an Invidious mirror `such as this one <https://invidious.site/>`__.
|
||||
|
||||
As mentioned above, the original Invidious has now closed. You can change the Invidious mirror that **Tartube** is using, if you like.
|
||||
|
||||
- Click **Edit > System preferences... > Operations > Downloads**
|
||||
- Enter a new mirror in the box
|
||||
- You can now watch a video by clicking its **Invidious** label
|
||||
|
||||
6.15 Filtering and finding videos
|
||||
---------------------------------
|
||||
@ -883,7 +931,7 @@ This is how to import the data into a different **Tartube** database.
|
||||
|
||||
**Tartube** can automatically extract the audio from its downloaded videos, if that's what you want.
|
||||
|
||||
The first step is to make sure that either FFmpeg or AVconv is installed on your system - see `6.4 Setting the location of FFmpeg / AVconv`_.
|
||||
The first step is to make sure that either FFmpeg or AVconv is installed on your system - see `6.4 Installing FFmpeg / AVConv`_.
|
||||
|
||||
The remaining steps are simple:
|
||||
|
||||
@ -1006,7 +1054,7 @@ Now click the **RSS feed** tab. Enter the address (URL) of the RSS feed in the b
|
||||
6.23 Detecting missing videos
|
||||
-----------------------------
|
||||
|
||||
Since v2.1.065, **Tartube** has been able to detect videos which you have downloaded, but which have since deleted by the original uploader.
|
||||
**Tartube** can detect videos you have downloaded, but which have been since deleted by the original uploader.
|
||||
|
||||
This feature is EXPERIMENTAL and may not work as intended.
|
||||
|
||||
@ -1017,7 +1065,79 @@ Having enabled detection, removed videos will appear in the **Missing Videos** f
|
||||
|
||||
**Tartube** only detects missing videos when checking/downloading whole channels or playlists. If you interrupt a download, no detection occurs.
|
||||
|
||||
You should note that enabling detection will disable the archive file used by youtube-dl (see `7.9 'Download all' button takes too long`_ ). Download operations may take longer as a result.
|
||||
6.24 More information about FFmpeg and AVConv
|
||||
---------------------------------------------
|
||||
|
||||
6.24.1 Using FFmpeg / AVConv with youtube-dl
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
If you explicitly set the location of the FFmpeg and/or AVConv executables, then those locations are passed on to youtube-dl when you check or download videos.
|
||||
|
||||
If *both* locations are set, only one of them is passed on. Usually, that's the location of FFmpeg. However, if you specify the **prefer_avconv** download option, then that is passed on, instead.
|
||||
|
||||
- Click **Edit > General download options...**
|
||||
- In the new window, if the **Show advanced download options** button is visible, click it
|
||||
- Now click the **Post-processing** tab
|
||||
- Click the **Prefer AVConv over FFmpeg** button to select it
|
||||
- Make sure the **Prefer FFmpeg over AVConv (default)** button is not selected
|
||||
- Click **OK** to apply your changes
|
||||
|
||||
For more information about download options, see `6.11 General download options`_.
|
||||
|
||||
6.24.2 Using FFmpeg directly
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
You can call FFmpeg directly, if you want to. (It only works on videos you have actually downloaded.)
|
||||
|
||||
This is useful for converting a video file from one format to another, and many other tasks.
|
||||
|
||||
- Click a video, or select several videos together
|
||||
- Right-click them and select **Process with FFmpeg...**
|
||||
- In the new dialogue window, select some FFmpeg options
|
||||
|
||||
.. image:: screenshots/example24.png
|
||||
:alt: The FFmpeg options window
|
||||
|
||||
6.24.3 Changing the filename
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The first three boxes allow you to change the video's filename. *This might take a very long time, if you don't add options in the other boxes, too.*
|
||||
|
||||
The first box allows you to add some text to the end of the filename, something like **modified**, perhaps.
|
||||
|
||||
The second and third boxes allow you to search and replace inside the filename.
|
||||
|
||||
In the box **If regex matches filename**, you can enter a regular expression (regex). If the pattern matches the filename, the matching portion is substituted for whatever you put in the box **...then apply substitution**.
|
||||
|
||||
If you're familiar with regular expressions, then this should need no further explanation: it's a perfectly ordinary regex substitution.
|
||||
|
||||
If not, then there are unlimited tutorials available online. Here's a simple example. To replace the word **rabbit** with **dinosaur**, in every filename that contains it, enter **rabbit** in the regex box and **dinosaur** in the substitution box.
|
||||
|
||||
6.24.4 Changing the video format
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Converting a video from one format to another is as simple as adding the text **avi** or **mkv** or any other valid video format to the box **Change file extension**,
|
||||
|
||||
6.24.5 FFmpeg command-line options
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The last box allows you to specify FFmpeg options directly. For example, to convert the framerate of some videos to 24 fps, enter the following text into the box at the bottom:
|
||||
|
||||
**-r 24**
|
||||
|
||||
6.25 Using youtube-dl forks
|
||||
---------------------------
|
||||
|
||||
`youtube-dl <https://youtube-dl.org/>`__ is open-source software, and there are a number of forks available (for example, `youtube-dlc <https://github.com/blackjack4494/youtube-dlc>`__).
|
||||
|
||||
If a youtube-dl fork is still compatible with the original, then **Tartube** can use it instead of the original.
|
||||
|
||||
- Click **Edit > System preferences... > youtube-dl**
|
||||
- In the box **youtube-dl compatible fork to use**, enter **youtube-dlc** (or the name of the fork)
|
||||
- Click **OK** to close the preferences window
|
||||
- Now click **Operations > Update youtube-dlc**, which will download (or update) the fork on your system
|
||||
|
||||
To switch back to using the original youtube-dl, just empty the same box.
|
||||
|
||||
7 Frequently-Asked Questions
|
||||
============================
|
||||
@ -1253,13 +1373,13 @@ A: See `6.20 Converting to audio`_
|
||||
|
||||
A: The solution to both problems is to install FFmpeg, and to set the output format correctly.
|
||||
|
||||
Firstly, make sure FFmpeg is installed on your system - see `6.4 Setting the location of FFmpeg / AVConv`_.
|
||||
Firstly, make sure FFmpeg is installed on your system - see `6.4 Installing FFmpeg / AVConv`_.
|
||||
|
||||
Secondly, set your desired output format. Open the Download options window (for example, click **Edit > General download options... > Formats > Preferred**). Add a format like **mp4** to the **List of preferred formats**, then add the same format to **If a merge is required after post-processing, output to this format**.
|
||||
|
||||
For some reason, youtube-dl ignores the download option unless the format is specified in both places. (You will see a warning if you forget.)
|
||||
|
||||
.. image:: screenshots/example24.png
|
||||
.. image:: screenshots/example25.png
|
||||
:alt: The Download options window
|
||||
|
||||
7.15 Too many folders in the main window
|
||||
@ -1403,15 +1523,14 @@ A: **Tartube** uses a set of stock icons wherever possible. If those icons are n
|
||||
|
||||
*Q: Tartube doesn't download video thumbnails any more! It used to work fine!*
|
||||
|
||||
A: In June 2020, **YouTube** changed its image format from **.jpg** to **.webp**. Unfortunately, most software (including the graphics libraries used by **Tartube**) don't support **.webp** images yet.
|
||||
A: In June 2020, **YouTube** changed its image format from **.jpg** to **.webp**. Unfortunately, most software (including the graphics libraries used by **Tartube**) don't support **.webp** images yet. Worse still, **YouTube** begain sending **.webp** thumbnails mislabelled as **.jpg**.
|
||||
|
||||
At the time of writing, a youtube-dl fix is expected. The fix is expected to convert **.webp** thumbnails back to **.jpg** thumbnails, after downloading them. The fix may require that `Ffmpeg <https://ffmpeg.org/>`__ is installed on your system.
|
||||
In September 2020, **Tartube** and **youtube-dl** added separate fixes for this problem. These fixes both depend on `FFmpeg <https://ffmpeg.org/>`__, so they won't work if FFmpeg is not installed on your system - see `6.4 Installing FFmpeg / AVConv`_.
|
||||
|
||||
Tartube can now look for and remove **.webp** fils automatically. You can use this procedure after the youtube-dl fix has been released.
|
||||
If you have already downloaded a lot of **.webp** images, you can ask **Tartube** to convert them back to **.jpg**. Once converted, they will be visible in the main window.
|
||||
|
||||
* Click **Operations > Tidy up files...**
|
||||
* In the dialogue window, click **Delete .webp/malformed .jpg files** to select it, then click the **OK** button
|
||||
* When the operation is completed, click the main **Check all** button to re-download thumbnails for all of your videos
|
||||
* In the dialogue window, click **Convert .webp files to .jpg using FFmpeg** to select it, then click the **OK** button
|
||||
|
||||
7.28 Tartube is not visible in the system tray
|
||||
----------------------------------------------
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 2.2 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 2.6 KiB After Width: | Height: | Size: 2.6 KiB |
BIN
icons/status/status_process_icon_64.png
Normal file
BIN
icons/status/status_process_icon_64.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 5.8 KiB |
BIN
icons/status/status_process_icon_xmas_64.png
Normal file
BIN
icons/status/status_process_icon_xmas_64.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 5.8 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.1 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 511 B |
File diff suppressed because it is too large
Load Diff
@ -1,4 +1,4 @@
|
||||
# Tartube v2.1.080 installer script for MS Windows
|
||||
# Tartube v2.2.0 installer script for MS Windows
|
||||
#
|
||||
# Copyright (C) 2019-2020 A S Lewis
|
||||
#
|
||||
@ -244,7 +244,7 @@
|
||||
|
||||
;Name and file
|
||||
Name "Tartube"
|
||||
OutFile "install-tartube-2.1.080-32bit.exe"
|
||||
OutFile "install-tartube-2.2.0-32bit.exe"
|
||||
|
||||
;Default installation folder
|
||||
InstallDir "$LOCALAPPDATA\Tartube"
|
||||
@ -347,7 +347,7 @@ Section "Tartube" SecClient
|
||||
# "Publisher" "A S Lewis"
|
||||
# WriteRegStr HKLM \
|
||||
# "Software\Microsoft\Windows\CurrentVersion\Uninstall\Tartube" \
|
||||
# "DisplayVersion" "2.1.080"
|
||||
# "DisplayVersion" "2.2.0"
|
||||
|
||||
# Create uninstaller
|
||||
WriteUninstaller "$INSTDIR\Uninstall.exe"
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Tartube v2.1.080 installer script for MS Windows
|
||||
# Tartube v2.2.0 installer script for MS Windows
|
||||
#
|
||||
# Copyright (C) 2019-2020 A S Lewis
|
||||
#
|
||||
@ -244,7 +244,7 @@
|
||||
|
||||
;Name and file
|
||||
Name "Tartube"
|
||||
OutFile "install-tartube-2.1.080-64bit.exe"
|
||||
OutFile "install-tartube-2.2.0-64bit.exe"
|
||||
|
||||
;Default installation folder
|
||||
InstallDir "$LOCALAPPDATA\Tartube"
|
||||
@ -347,7 +347,7 @@ Section "Tartube" SecClient
|
||||
# "Publisher" "A S Lewis"
|
||||
# WriteRegStr HKLM \
|
||||
# "Software\Microsoft\Windows\CurrentVersion\Uninstall\Tartube" \
|
||||
# "DisplayVersion" "2.1.080"
|
||||
# "DisplayVersion" "2.2.0"
|
||||
|
||||
# Create uninstaller
|
||||
WriteUninstaller "$INSTDIR\Uninstall.exe"
|
||||
|
||||
@ -42,8 +42,8 @@ import mainapp
|
||||
|
||||
# 'Global' variables
|
||||
__packagename__ = 'tartube'
|
||||
__version__ = '2.1.080'
|
||||
__date__ = '13 Aug 2020'
|
||||
__version__ = '2.2.0'
|
||||
__date__ = '30 Sep 2020'
|
||||
__copyright__ = 'Copyright \xa9 2019-2020 A S Lewis'
|
||||
__license__ = """
|
||||
Copyright \xa9 2019-2020 A S Lewis.
|
||||
|
||||
@ -42,8 +42,8 @@ import mainapp
|
||||
|
||||
# 'Global' variables
|
||||
__packagename__ = 'tartube'
|
||||
__version__ = '2.1.080'
|
||||
__date__ = '13 Aug 2020'
|
||||
__version__ = '2.2.0'
|
||||
__date__ = '30 Sep 2020'
|
||||
__copyright__ = 'Copyright \xa9 2019-2020 A S Lewis'
|
||||
__license__ = """
|
||||
Copyright \xa9 2019-2020 A S Lewis.
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
.TH man 1 "13 Aug 2020" "2.1.080" "tartube man page"
|
||||
.TH man 1 "30 Sep 2020" "2.2.0" "tartube man page"
|
||||
.SH NAME
|
||||
tartube \- GUI front-end for youtube-dl
|
||||
.SH SYNOPSIS
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[Desktop Entry]
|
||||
Name=Tartube
|
||||
Version=2.1.080
|
||||
Version=2.2.0
|
||||
Exec=tartube
|
||||
Icon=tartube
|
||||
Type=Application
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 47 KiB After Width: | Height: | Size: 49 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 23 KiB After Width: | Height: | Size: 22 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 29 KiB After Width: | Height: | Size: 28 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 54 KiB After Width: | Height: | Size: 49 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 54 KiB After Width: | Height: | Size: 49 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 54 KiB After Width: | Height: | Size: 49 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 182 KiB After Width: | Height: | Size: 172 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 54 KiB After Width: | Height: | Size: 47 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 202 KiB After Width: | Height: | Size: 190 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 56 KiB After Width: | Height: | Size: 56 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 53 KiB After Width: | Height: | Size: 27 KiB |
BIN
screenshots/example25.png
Normal file
BIN
screenshots/example25.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 53 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 88 KiB After Width: | Height: | Size: 77 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 30 KiB After Width: | Height: | Size: 30 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 20 KiB After Width: | Height: | Size: 31 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 39 KiB After Width: | Height: | Size: 42 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 29 KiB After Width: | Height: | Size: 30 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 58 KiB After Width: | Height: | Size: 60 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 249 KiB After Width: | Height: | Size: 258 KiB |
7
setup.py
7
setup.py
@ -44,6 +44,8 @@ unsubscribe people from their favourite channels and/or deliberately
|
||||
conceal videos that they don't like. Tartube won't do any of those things
|
||||
- Tartube can, in some circumstances, see videos that are region-blocked
|
||||
and/or age-restricted
|
||||
|
||||
Note for PyPI users: Tartube should be installed with: pip3 install tartube
|
||||
"""
|
||||
|
||||
alt_description = """
|
||||
@ -145,7 +147,7 @@ for path in glob.glob('sounds/*'):
|
||||
# Setup
|
||||
setuptools.setup(
|
||||
name='tartube',
|
||||
version='2.1.080',
|
||||
version='2.2.0',
|
||||
description='GUI front-end for youtube-dl',
|
||||
long_description=long_description,
|
||||
long_description_content_type='text/plain',
|
||||
@ -171,7 +173,8 @@ setuptools.setup(
|
||||
exclude=('docs', 'icons', 'nsis', 'tests'),
|
||||
),
|
||||
include_package_data=True,
|
||||
python_requires='>=3.0, <4',
|
||||
# python_requires='>=3.0, <4',
|
||||
python_requires='>=3.0',
|
||||
install_requires=['feedparser', 'pgi', 'playsound', 'requests'],
|
||||
scripts=[script_exec],
|
||||
project_urls={
|
||||
|
||||
1160
tartube/config.py
1160
tartube/config.py
File diff suppressed because it is too large
Load Diff
@ -2356,8 +2356,8 @@ class VideoDownloader(object):
|
||||
0,
|
||||
app_obj.system_error,
|
||||
302,
|
||||
'Enforced timeout on youtube-dl because it took too long' \
|
||||
+ ' to fetch a video\'s JSON data',
|
||||
'Enforced timeout because downloader took too long to' \
|
||||
+ ' fetch a video\'s JSON data',
|
||||
)
|
||||
|
||||
# Stop this video downloader, if required to do so, having just
|
||||
@ -2580,6 +2580,78 @@ class VideoDownloader(object):
|
||||
self.stderr_reader.join()
|
||||
|
||||
|
||||
def compile_mini_options_dict(self, options_manager_obj):
|
||||
|
||||
"""Called by self.confirm_new_video() and .confirm_old_video().
|
||||
|
||||
Compiles a dictionary containing a subset of download options from the
|
||||
specified options.OptionsManager object, to be passed on to
|
||||
mainapp.TartubeApp.announce_video_download().
|
||||
|
||||
Args:
|
||||
|
||||
options_manager_obj (options.OptionsManager): The options manager
|
||||
for this download
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('dld 2457 compile_mini_options_dict')
|
||||
|
||||
mini_options_dict = {
|
||||
'keep_description': \
|
||||
options_manager_obj.options_dict['keep_description'],
|
||||
'keep_info': \
|
||||
options_manager_obj.options_dict['keep_info'],
|
||||
'keep_annotations': \
|
||||
options_manager_obj.options_dict['keep_annotations'],
|
||||
'keep_thumbnail': \
|
||||
options_manager_obj.options_dict['keep_thumbnail'],
|
||||
'move_description': \
|
||||
options_manager_obj.options_dict['move_description'],
|
||||
'move_info': \
|
||||
options_manager_obj.options_dict['move_info'],
|
||||
'move_annotations': \
|
||||
options_manager_obj.options_dict['move_annotations'],
|
||||
'move_thumbnail': \
|
||||
options_manager_obj.options_dict['move_thumbnail'],
|
||||
}
|
||||
|
||||
return mini_options_dict
|
||||
|
||||
|
||||
def confirm_archived_video(self, filename):
|
||||
|
||||
"""Called by self.extract_stdout_data().
|
||||
|
||||
A modified version of self.confirm_old_video(), called when
|
||||
youtube-dl's 'has already been recorded in archive' message is detected
|
||||
(but only when checking for missing videos).
|
||||
|
||||
Tries to find a match for the video name and, if one is found, marks it
|
||||
as not missing.
|
||||
|
||||
Args:
|
||||
|
||||
filename (str): The video name, which should match the .name of a
|
||||
media.Video object in self.missing_video_check_list
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('dld 2458 confirm_archived_video')
|
||||
|
||||
# Create shortcut variables (for convenience)
|
||||
app_obj = self.download_manager_obj.app_obj
|
||||
media_data_obj = self.download_item_obj.media_data_obj
|
||||
|
||||
# media_data_obj is a media.Channel or media.Playlist object. Check its
|
||||
# child objects, looking for a matching video
|
||||
match_obj = media_data_obj.find_matching_video(app_obj, filename)
|
||||
if match_obj and match_obj in self.missing_video_check_list:
|
||||
self.missing_video_check_list.remove(match_obj)
|
||||
|
||||
|
||||
def confirm_new_video(self, dir_path, filename, extension):
|
||||
|
||||
"""Called by self.extract_stdout_data().
|
||||
@ -2653,19 +2725,15 @@ class VideoDownloader(object):
|
||||
or isinstance(video_obj.parent_obj, media.Playlist):
|
||||
video_obj.set_index(self.video_num)
|
||||
|
||||
# Fetch the options.OptionsManager object used for this download
|
||||
options_manager_obj = self.download_worker_obj.options_manager_obj
|
||||
|
||||
# Update the main window
|
||||
GObject.timeout_add(
|
||||
0,
|
||||
app_obj.announce_video_download,
|
||||
self.download_item_obj,
|
||||
video_obj,
|
||||
options_manager_obj.options_dict['keep_description'],
|
||||
options_manager_obj.options_dict['keep_info'],
|
||||
options_manager_obj.options_dict['keep_annotations'],
|
||||
options_manager_obj.options_dict['keep_thumbnail'],
|
||||
self.compile_mini_options_dict(
|
||||
self.download_worker_obj.options_manager_obj,
|
||||
),
|
||||
)
|
||||
|
||||
# Register the download with DownloadManager, so that download
|
||||
@ -2782,11 +2850,6 @@ class VideoDownloader(object):
|
||||
extension,
|
||||
)
|
||||
|
||||
# Fetch the options.OptionsManager object used for this
|
||||
# download
|
||||
options_manager_obj \
|
||||
= self.download_worker_obj.options_manager_obj
|
||||
|
||||
# Update the main window
|
||||
if media_data_obj.master_dbid != media_data_obj.dbid:
|
||||
|
||||
@ -2809,10 +2872,9 @@ class VideoDownloader(object):
|
||||
app_obj.announce_video_download,
|
||||
self.download_item_obj,
|
||||
video_obj,
|
||||
options_manager_obj.options_dict['keep_description'],
|
||||
options_manager_obj.options_dict['keep_info'],
|
||||
options_manager_obj.options_dict['keep_annotations'],
|
||||
options_manager_obj.options_dict['keep_thumbnail'],
|
||||
self.compile_mini_options_dict(
|
||||
self.download_worker_obj.options_manager_obj,
|
||||
),
|
||||
)
|
||||
|
||||
# This VideoDownloader can now stop, if required to do so after a video
|
||||
@ -3118,13 +3180,17 @@ class VideoDownloader(object):
|
||||
|
||||
# Deal with the video description, JSON data and thumbnail, according
|
||||
# to the settings in options.OptionsManager
|
||||
options_dict =self.download_worker_obj.options_manager_obj.options_dict
|
||||
options_dict \
|
||||
= self.download_worker_obj.options_manager_obj.options_dict
|
||||
|
||||
if descrip and options_dict['write_description']:
|
||||
|
||||
descrip_path = os.path.abspath(
|
||||
os.path.join(path, filename + '.description'),
|
||||
)
|
||||
|
||||
if not options_dict['sim_keep_description']:
|
||||
|
||||
descrip_path = utils.convert_path_to_temp(
|
||||
app_obj,
|
||||
descrip_path,
|
||||
@ -3134,33 +3200,58 @@ class VideoDownloader(object):
|
||||
# do anything if the call returned None because of a filesystem
|
||||
# error)
|
||||
if descrip_path is not None and not os.path.isfile(descrip_path):
|
||||
|
||||
try:
|
||||
fh = open(descrip_path, 'wb')
|
||||
fh.write(descrip.encode('utf-8'))
|
||||
fh.close()
|
||||
|
||||
if options_dict['move_description']:
|
||||
utils.move_metadata_to_subdir(
|
||||
app_obj,
|
||||
video_obj,
|
||||
'.description',
|
||||
)
|
||||
|
||||
except:
|
||||
pass
|
||||
|
||||
if options_dict['write_info']:
|
||||
|
||||
json_path = os.path.abspath(
|
||||
os.path.join(path, filename + '.info.json'),
|
||||
)
|
||||
|
||||
if not options_dict['sim_keep_info']:
|
||||
json_path = utils.convert_path_to_temp(app_obj, json_path)
|
||||
|
||||
if json_path is not None and not os.path.isfile(json_path):
|
||||
|
||||
try:
|
||||
with open(json_path, 'w') as outfile:
|
||||
json.dump(json_dict, outfile, indent=4)
|
||||
|
||||
if options_dict['move_info']:
|
||||
utils.move_metadata_to_subdir(
|
||||
app_obj,
|
||||
video_obj,
|
||||
'.info.json',
|
||||
)
|
||||
|
||||
except:
|
||||
pass
|
||||
|
||||
if options_dict['write_annotations']:
|
||||
xml_path = os.path.abspath(
|
||||
os.path.join(path, filename + '.annotations.xml'),
|
||||
)
|
||||
if not options_dict['sim_keep_annotations']:
|
||||
xml_path = utils.convert_path_to_temp(app_obj, xml_path)
|
||||
# v2.1.101 - Annotations were removed by YouTube in 2019, so this
|
||||
# feature is not available, and will not be available until the
|
||||
# authors have some annotations to test
|
||||
# if options_dict['write_annotations']:
|
||||
#
|
||||
# xml_path = os.path.abspath(
|
||||
# os.path.join(path, filename + '.annotations.xml'),
|
||||
# )
|
||||
#
|
||||
# if not options_dict['sim_keep_annotations']:
|
||||
# xml_path = utils.convert_path_to_temp(app_obj, xml_path)
|
||||
|
||||
if thumbnail and options_dict['write_thumbnail']:
|
||||
|
||||
@ -3181,8 +3272,8 @@ class VideoDownloader(object):
|
||||
|
||||
if thumb_path is not None and not os.path.isfile(thumb_path):
|
||||
|
||||
# v2.0.013 The requets module fails if the connection drops
|
||||
# v1.2.006 Wiriting the file fails if the directory specified
|
||||
# v2.0.013 The requests module fails if the connection drops
|
||||
# v1.2.006 Writing the file fails if the directory specified
|
||||
# by thumb_path doesn't exist
|
||||
# Use 'try' so that neither problem is fatal
|
||||
try:
|
||||
@ -3193,9 +3284,31 @@ class VideoDownloader(object):
|
||||
except:
|
||||
pass
|
||||
|
||||
# Convert .webp thumbnails to .jpg, if required
|
||||
thumb_path = utils.find_thumbnail_webp(app_obj, video_obj)
|
||||
if thumb_path is not None \
|
||||
and not app_obj.ffmpeg_fail_flag \
|
||||
and app_obj.ffmpeg_convert_webp_flag \
|
||||
and not app_obj.ffmpeg_manager_obj.convert_webp(thumb_path):
|
||||
|
||||
app_obj.set_ffmpeg_fail_flag(True)
|
||||
GObject.timeout_add(
|
||||
0,
|
||||
app_obj.system_error,
|
||||
307,
|
||||
app_obj.ffmpeg_fail_msg,
|
||||
)
|
||||
|
||||
# Move to the sub-directory, if required
|
||||
if options_dict['move_thumbnail']:
|
||||
|
||||
utils.move_thumbnail_to_subdir(app_obj, video_obj)
|
||||
|
||||
# If a new media.Video object was created (or if a video whose name is
|
||||
# unknown, now has a name), add a line to the Results List, as well
|
||||
# as updating the Video Catalogue
|
||||
# The True argument passes on the download options 'move_description',
|
||||
# etc, but not 'keep_description', etc
|
||||
if update_results_flag:
|
||||
|
||||
GObject.timeout_add(
|
||||
@ -3203,6 +3316,10 @@ class VideoDownloader(object):
|
||||
app_obj.announce_video_download,
|
||||
self.download_item_obj,
|
||||
video_obj,
|
||||
# No call to self.compile_mini_options_dict, because this
|
||||
# function deals with download options like
|
||||
# 'move_description' by itself
|
||||
{},
|
||||
)
|
||||
|
||||
else:
|
||||
@ -3510,6 +3627,9 @@ class VideoDownloader(object):
|
||||
stdout_with_spaces_list = stdout.split(' ')
|
||||
stdout_list = stdout.split()
|
||||
|
||||
# (Flag set to True when self.confirm_new_video(), etc, are called)
|
||||
confirm_flag = False
|
||||
|
||||
# Extract the data
|
||||
stdout_list[0] = stdout_list[0].lstrip('\r')
|
||||
if stdout_list[0] == '[download]':
|
||||
@ -3556,6 +3676,7 @@ class VideoDownloader(object):
|
||||
)
|
||||
|
||||
self.reset_temp_destination()
|
||||
confirm_flag = True
|
||||
|
||||
# Get playlist information (when downloading a channel or a
|
||||
# playlist, this line is received once per video)
|
||||
@ -3591,11 +3712,26 @@ class VideoDownloader(object):
|
||||
self.reset_temp_destination()
|
||||
|
||||
self.confirm_old_video(path, filename, extension)
|
||||
confirm_flag = True
|
||||
|
||||
# Get filesize abort status
|
||||
if stdout_list[-1] == 'Aborting.':
|
||||
dl_stat_dict['status'] = formats.ERROR_STAGE_ABORT
|
||||
|
||||
# When checking for missing videos, respond to the 'has already
|
||||
# been recorded in archive' message (which is otherwise ignored)
|
||||
if not confirm_flag \
|
||||
and self.missing_video_check_list:
|
||||
|
||||
match = re.search(
|
||||
r'^\[download\]\s(.*)\shas already been recorded in' \
|
||||
+ ' archive$',
|
||||
stdout,
|
||||
)
|
||||
|
||||
if match:
|
||||
self.confirm_archived_video(match.group(1))
|
||||
|
||||
elif stdout_list[0] == '[hlsnative]':
|
||||
|
||||
# Get information from the native HLS extractor (see
|
||||
@ -4279,7 +4415,7 @@ class JSONFetcher(object):
|
||||
|
||||
# Convert a youtube-dl path beginning with ~ (not on MS Windows)
|
||||
# (code copied from utils.generate_system_cmd() )
|
||||
ytdl_path = app_obj.ytdl_path
|
||||
ytdl_path = app_obj.check_downloader(app_obj.ytdl_path)
|
||||
if os.name != 'nt':
|
||||
ytdl_path = re.sub('^\~', os.path.expanduser('~'), ytdl_path)
|
||||
|
||||
@ -4389,6 +4525,15 @@ class JSONFetcher(object):
|
||||
local_thumb_path,
|
||||
)
|
||||
|
||||
elif options_obj.options_dict['move_thumbnail']:
|
||||
local_thumb_path = os.path.abspath(
|
||||
os.path.join(
|
||||
self.container_obj.get_actual_dir(app_obj),
|
||||
app_obj.thumbs_sub_dir,
|
||||
self.video_name + remote_ext,
|
||||
)
|
||||
)
|
||||
|
||||
if local_thumb_path:
|
||||
try:
|
||||
request_obj = requests.get(self.video_thumb_source)
|
||||
@ -4398,6 +4543,22 @@ class JSONFetcher(object):
|
||||
except:
|
||||
pass
|
||||
|
||||
# Convert .webp thumbnails to .jpg, if required
|
||||
if local_thumb_path is not None \
|
||||
and not app_obj.ffmpeg_fail_flag \
|
||||
and app_obj.ffmpeg_convert_webp_flag \
|
||||
and not app_obj.ffmpeg_manager_obj.convert_webp(
|
||||
local_thumb_path
|
||||
):
|
||||
app_obj.set_ffmpeg_fail_flag(True)
|
||||
GObject.timeout_add(
|
||||
0,
|
||||
app_obj.system_error,
|
||||
308,
|
||||
app_obj.ffmpeg_fail_msg,
|
||||
)
|
||||
|
||||
|
||||
|
||||
def close(self):
|
||||
|
||||
@ -4816,7 +4977,7 @@ class MiniJSONFetcher(object):
|
||||
|
||||
# Convert a youtube-dl path beginning with ~ (not on MS Windows)
|
||||
# (code copied from utils.generate_system_cmd() )
|
||||
ytdl_path = app_obj.ytdl_path
|
||||
ytdl_path = app_obj.check_downloader(app_obj.ytdl_path)
|
||||
if os.name != 'nt':
|
||||
ytdl_path = re.sub('^\~', os.path.expanduser('~'), ytdl_path)
|
||||
|
||||
@ -5023,7 +5184,7 @@ class MiniJSONFetcher(object):
|
||||
stdout (str): A string of JSON data as it was received from
|
||||
youtube-dl (and starting with the character { )
|
||||
|
||||
Return values:
|
||||
Returns:
|
||||
|
||||
The JSON data, converted into a Python dictionary
|
||||
|
||||
|
||||
405
tartube/ffmpeg.py
Normal file
405
tartube/ffmpeg.py
Normal file
@ -0,0 +1,405 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
# Copyright (C) 2019-2020 A S Lewis
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify it under
|
||||
# the terms of the GNU General Public License as published by the Free Software
|
||||
# Foundation, either version 3 of the License, or (at your option) any later
|
||||
# version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
|
||||
# details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License along with
|
||||
# this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
|
||||
"""FFmpeg manager classes."""
|
||||
|
||||
|
||||
# Import Gtk modules
|
||||
# ...
|
||||
|
||||
|
||||
# Import other modules
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
|
||||
# Import our modules
|
||||
# ...
|
||||
|
||||
|
||||
# Classes
|
||||
|
||||
|
||||
class FFmpegManager(object):
|
||||
|
||||
"""Called by mainapp.TartubeApp.__init__().
|
||||
|
||||
Python class to manage calls to FFmpeg that Tartube wants to make,
|
||||
independently of youtube-dl.
|
||||
|
||||
Most of the code in this file has been updated from youtube-dl itself.
|
||||
|
||||
Args:
|
||||
|
||||
app_obj (mainapp.TartubeApp): The main application object
|
||||
|
||||
"""
|
||||
|
||||
|
||||
# Standard class methods
|
||||
|
||||
|
||||
def __init__(self, app_obj):
|
||||
|
||||
|
||||
super(FFmpegManager, self).__init__()
|
||||
|
||||
# IV list - class objects
|
||||
# -----------------------
|
||||
# The main application
|
||||
self.app_obj = app_obj
|
||||
|
||||
|
||||
# Public class methods
|
||||
|
||||
|
||||
def convert_webp(self, thumbnail_filename):
|
||||
|
||||
"""Called by mainapp.TartubeApp.update_video_when_file_found(),
|
||||
downloads.VideoDownloader.confirm_sim_video() and
|
||||
downloads.JSONFetcher.do_fetch().
|
||||
|
||||
Adapted from youtube-dl/youtube-dl/postprocessor/embedthumbnail.py.
|
||||
|
||||
In June 2020, YouTube changed its image format from .jpg to .webp.
|
||||
Unfortunately, the Gtk library doesn't support that format.
|
||||
|
||||
Worse still, YouTube also began sending .webp thumbnails mislabelled as
|
||||
.jpg.
|
||||
|
||||
In response, in September 2020 youtube-dl implemented a fix for
|
||||
embedded thumbnails, using FFmpeg to convert .webp to .jpg. That code
|
||||
has been adapted here, so that YouTube thumbnails can be converted and
|
||||
made visible in the main window again.
|
||||
|
||||
Args:
|
||||
|
||||
thumbnail_filename (str): Full path to the webp file to be
|
||||
converted to jpg
|
||||
|
||||
Returns:
|
||||
|
||||
False if an attempted conversion fails, or True otherwise
|
||||
(including when no conversion is attempted)
|
||||
|
||||
"""
|
||||
|
||||
# Sanity check
|
||||
if not os.path.isfile(thumbnail_filename) \
|
||||
or self.app_obj.ffmpeg_fail_flag:
|
||||
return True
|
||||
|
||||
# Correct extension for .webp files with the wrong extension
|
||||
# (youtube-dl #25687, #25717)
|
||||
_, thumbnail_ext = os.path.splitext(thumbnail_filename)
|
||||
if thumbnail_ext:
|
||||
|
||||
# Remove the initial full stop
|
||||
thumbnail_ext = thumbnail_ext[1:].lower()
|
||||
|
||||
if thumbnail_ext != 'webp' and self.is_webp(thumbnail_filename):
|
||||
|
||||
# .webp mislabelled as .jpg
|
||||
thumbnail_webp_filename = self.replace_extension(
|
||||
thumbnail_filename,
|
||||
'webp',
|
||||
)
|
||||
|
||||
os.rename(thumbnail_filename, thumbnail_webp_filename)
|
||||
thumbnail_filename = thumbnail_webp_filename
|
||||
thumbnail_ext = 'webp'
|
||||
|
||||
# Convert unsupported thumbnail formats to JPEG
|
||||
# (youtube-dl #25687, #25717)
|
||||
if thumbnail_ext not in ['jpg', 'png']:
|
||||
|
||||
# NB: % is supposed to be escaped with %% but this does not work
|
||||
# for input files so working around with standard substitution
|
||||
escaped_thumbnail_filename = thumbnail_filename.replace('%', '#')
|
||||
os.rename(thumbnail_filename, escaped_thumbnail_filename)
|
||||
escaped_thumbnail_jpg_filename = self.replace_extension(
|
||||
escaped_thumbnail_filename,
|
||||
'jpg',
|
||||
)
|
||||
|
||||
# Run FFmpeg, which eturns a list in the form
|
||||
# (success_flag, optional_message)
|
||||
result_list = self.run_ffmpeg(
|
||||
escaped_thumbnail_filename,
|
||||
escaped_thumbnail_jpg_filename,
|
||||
['-bsf:v', 'mjpeg2jpeg'],
|
||||
)
|
||||
|
||||
if not result_list or not result_list[0]:
|
||||
|
||||
# Conversion failed; most likely because FFmpeg is not
|
||||
# installed
|
||||
# Rename back to unescaped
|
||||
os.rename(escaped_thumbnail_filename, thumbnail_filename)
|
||||
|
||||
return False
|
||||
|
||||
else:
|
||||
|
||||
# Conversion succeeded
|
||||
os.remove(escaped_thumbnail_filename)
|
||||
thumbnail_jpg_filename = self.replace_extension(
|
||||
thumbnail_filename,
|
||||
'jpg',
|
||||
)
|
||||
|
||||
# Rename back to unescaped for further processing
|
||||
os.rename(
|
||||
escaped_thumbnail_jpg_filename,
|
||||
thumbnail_jpg_filename
|
||||
)
|
||||
|
||||
# Procedure complete
|
||||
return True
|
||||
|
||||
|
||||
def _ffmpeg_filename_argument(self, path):
|
||||
|
||||
"""Called by self.run_ffmpeg_multiple_files().
|
||||
|
||||
Adapted from youtube-dl/youtube-dl/postprocessor/ffmpeg.py.
|
||||
|
||||
Returns a filename in a format that won't confuse FFmpeg.
|
||||
|
||||
Args:
|
||||
|
||||
path (str): The full path to a file to be processed by FFmpeg
|
||||
|
||||
Returns:
|
||||
|
||||
The modified string
|
||||
|
||||
"""
|
||||
|
||||
# Always use 'file:' because the filename may contain ':' (ffmpeg
|
||||
# interprets that as a protocol) or can start with '-' (-- is broken
|
||||
# in ffmpeg, see https://ffmpeg.org/trac/ffmpeg/ticket/2127 for
|
||||
# details)
|
||||
# Also leave '-' intact in order not to break streaming to stdout
|
||||
return 'file:' + path if path != '-' else path
|
||||
|
||||
|
||||
def get_executable(self):
|
||||
|
||||
"""Called by self.run_ffmpeg_multiple_files().
|
||||
|
||||
Not adapted from youtube-dl.
|
||||
|
||||
Returns the path to the FFmpeg executable, which the user may have
|
||||
specified themselves. If not, assume ffmpeg is in the system path.
|
||||
|
||||
Returns:
|
||||
|
||||
The path to the executable
|
||||
|
||||
"""
|
||||
|
||||
if self.app_obj.ffmpeg_path:
|
||||
return self.app_obj.ffmpeg_path
|
||||
else:
|
||||
return 'ffmpeg'
|
||||
|
||||
|
||||
def is_webp(self, path):
|
||||
|
||||
"""Called by self.convert_webp() and utils.find_thumbnail_webp().
|
||||
|
||||
Adapted from youtube-dl/youtube-dl/postprocessor/embedthumbnail.py.
|
||||
|
||||
Tests whether a file is a .webp file (perhaps mislabelled as a .jpg
|
||||
file).
|
||||
|
||||
Args:
|
||||
|
||||
path (str): The full path to a file to be processed by FFmpeg
|
||||
|
||||
"""
|
||||
|
||||
with open(path, 'rb') as fh:
|
||||
data = fh.read(12)
|
||||
|
||||
return data[0:4] == b'RIFF' and data[8:] == b'WEBP'
|
||||
|
||||
|
||||
def replace_extension(self, path, ext, expected_real_ext=None):
|
||||
|
||||
"""Called by self.convert_webp().
|
||||
|
||||
Adapted from youtube-dl/youtube-dl/utils.py.
|
||||
|
||||
Given the full path to a file, replaces the extension, and returns the
|
||||
modified path.
|
||||
|
||||
Args:
|
||||
|
||||
path (str): The full path to a file
|
||||
|
||||
ext (str): The new file extension
|
||||
|
||||
expected_real_ext (str): Not used by Tartube
|
||||
|
||||
Returns:
|
||||
|
||||
The modified path
|
||||
|
||||
"""
|
||||
|
||||
name, real_ext = os.path.splitext(path)
|
||||
|
||||
return '{0}.{1}'.format(
|
||||
name if not expected_real_ext \
|
||||
or real_ext[1:] == expected_real_ext \
|
||||
else path,
|
||||
ext,
|
||||
)
|
||||
|
||||
|
||||
def run_ffmpeg(self, input_path, out_path, opt_list, test_flag=False):
|
||||
|
||||
"""Can be called by anything.
|
||||
|
||||
Currently called by self.convert_webp() and
|
||||
process.ProcessManager.process_video().
|
||||
|
||||
Adapted from youtube-dl/youtube-dl/postprocessor/ffmpeg.py.
|
||||
|
||||
self.run_ffmpeg_multiple_files() expects a list of files. Pass on
|
||||
this function's parameters in the expected format.
|
||||
|
||||
Args:
|
||||
|
||||
input_path (str): Full path to a file to be processed by FFmpeg
|
||||
|
||||
out_path (str): Full path to FFmpeg's output file
|
||||
|
||||
opt_list (list): List of FFmpeg command line options (may be an
|
||||
empty list)
|
||||
|
||||
test_flag (bool): If True, just returns the FFmpeg system command,
|
||||
rather than executing it
|
||||
|
||||
"""
|
||||
|
||||
return self.run_ffmpeg_multiple_files(
|
||||
[ input_path ],
|
||||
out_path,
|
||||
opt_list,
|
||||
test_flag,
|
||||
)
|
||||
|
||||
|
||||
def run_ffmpeg_multiple_files(self, input_path_list, out_path, opt_list, \
|
||||
test_flag=False):
|
||||
|
||||
"""Can be called by anything.
|
||||
|
||||
Currently called by self.run_ffmpeg().
|
||||
|
||||
Adapted from youtube-dl/youtube-dl/postprocessor/ffmpeg.py.
|
||||
|
||||
Prepares the FFmpeg system command, and then executes it.
|
||||
|
||||
Args:
|
||||
|
||||
input_path_list (list): List of full paths to files to be
|
||||
processed by FFmpeg. At the moment, Tartube only processes one
|
||||
file at a time
|
||||
|
||||
out_path (str): Full path to FFmpeg's output file
|
||||
|
||||
opt_list (list): List of FFmpeg command line options (may be an
|
||||
empty list)
|
||||
|
||||
test_flag (bool): If True, just returns the FFmpeg system command,
|
||||
rather than executing it
|
||||
|
||||
Return values:
|
||||
|
||||
Returns a list of two items, in the form
|
||||
(success_flag, optional_message)
|
||||
|
||||
"""
|
||||
|
||||
# Get the modification time for the oldest file
|
||||
oldest_mtime = min(os.stat(path).st_mtime for path in input_path_list)
|
||||
|
||||
# Prepare the system command
|
||||
files_cmd_list = []
|
||||
for path in input_path_list:
|
||||
files_cmd_list.extend(['-i', self._ffmpeg_filename_argument(path)])
|
||||
|
||||
cmd_list = [self.get_executable(), '-y']
|
||||
cmd_list += ['-loglevel', 'repeat+info']
|
||||
cmd_list += (
|
||||
files_cmd_list
|
||||
+ opt_list
|
||||
+ [self._ffmpeg_filename_argument(out_path)]
|
||||
)
|
||||
|
||||
# Return the system command only, if required
|
||||
if test_flag:
|
||||
return [ True, cmd_list ]
|
||||
|
||||
# Execute the system command in a subprocess
|
||||
try:
|
||||
p = subprocess.Popen(
|
||||
cmd_list,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
stdin=subprocess.PIPE,
|
||||
)
|
||||
|
||||
except:
|
||||
# If FFmpeg is not installed on the user's system, this is the
|
||||
# result
|
||||
return [ False, 'Could not find FFmpeg' ]
|
||||
|
||||
stdout, stderr = p.communicate()
|
||||
if p.returncode != 0:
|
||||
stderr = stderr.decode('utf-8', 'replace')
|
||||
return [ False, stderr.strip().split('\n')[-1] ]
|
||||
|
||||
else:
|
||||
return [ self.try_utime(out_path, oldest_mtime, oldest_mtime), '' ]
|
||||
|
||||
|
||||
def try_utime(self, path, atime, mtime):
|
||||
|
||||
"""Called by self.run_ffmpeg_multiple_files().
|
||||
|
||||
Adapted from youtube-dl/youtube-dl/postprocessor/common.py.
|
||||
|
||||
Return values:
|
||||
|
||||
True on success, False on failure
|
||||
|
||||
"""
|
||||
|
||||
try:
|
||||
os.utime(path, (atime, mtime))
|
||||
return True
|
||||
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
@ -136,7 +136,8 @@ class FileManager(threading.Thread):
|
||||
|
||||
Returns:
|
||||
|
||||
A GdkPixbuf, or None if the file is missing or can't be loaded
|
||||
A GdkPixbuf (as a tuple), or None if the file is missing or can't
|
||||
be loaded
|
||||
|
||||
"""
|
||||
|
||||
@ -144,6 +145,7 @@ class FileManager(threading.Thread):
|
||||
return None
|
||||
|
||||
try:
|
||||
# (Returns a tuple)
|
||||
pixbuf = GdkPixbuf.Pixbuf.new_from_file(full_path)
|
||||
except:
|
||||
return None
|
||||
|
||||
@ -344,6 +344,8 @@ while audio_setup_list:
|
||||
# (Used for detecting video thumbnails. Unfortunately Gtk can't display .webp
|
||||
# files yet)
|
||||
IMAGE_FORMAT_LIST = ['.jpg', '.png', '.gif']
|
||||
# (The same list including .webp, for any code that needs it)
|
||||
IMAGE_FORMAT_EXT_LIST = ['.jpg', '.png', '.gif', '.webp']
|
||||
|
||||
FILE_SIZE_UNIT_LIST = [
|
||||
['Bytes', ''],
|
||||
@ -580,6 +582,7 @@ if not xmas_flag:
|
||||
'info_icon': 'status_info_icon_64.png',
|
||||
'tidy_icon': 'status_tidy_icon_64.png',
|
||||
'livestream_icon': 'status_livestream_icon_64.png',
|
||||
'process_icon': 'status_process_icon_64.png',
|
||||
}
|
||||
else:
|
||||
STATUS_ICON_DICT = {
|
||||
@ -591,6 +594,7 @@ else:
|
||||
'info_icon': 'status_info_icon_xmas_64.png',
|
||||
'tidy_icon': 'status_tidy_icon_xmas_64.png',
|
||||
'livestream_icon': 'status_livestream_icon_xmas_64.png',
|
||||
'process_icon': 'status_process_icon_xmas_64.png',
|
||||
}
|
||||
|
||||
TOOLBAR_ICON_DICT = {
|
||||
@ -610,8 +614,6 @@ TOOLBAR_ICON_DICT = {
|
||||
'tool_stop_small': 'stop_small.png',
|
||||
'tool_switch_large': 'switch_large.png',
|
||||
'tool_switch_small': 'switch_small.png',
|
||||
'tool_test_large': 'test_large.png',
|
||||
'tool_test_small': 'test_small.png',
|
||||
'tool_video_large': 'video_large.png',
|
||||
'tool_video_small': 'video_small.png',
|
||||
}
|
||||
|
||||
@ -42,10 +42,6 @@ import utils
|
||||
from mainapp import _
|
||||
|
||||
|
||||
# Debugging flag (calls utils.debug_time at the start of every function)
|
||||
DEBUG_FUNC_FLAG = False
|
||||
|
||||
|
||||
# Classes
|
||||
|
||||
|
||||
@ -98,9 +94,6 @@ class InfoManager(threading.Thread):
|
||||
def __init__(self, app_obj, info_type, media_data_obj, url_string,
|
||||
options_string):
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('iop 102 __init__')
|
||||
|
||||
super(InfoManager, self).__init__()
|
||||
|
||||
# IV list - class objects
|
||||
@ -176,14 +169,11 @@ class InfoManager(threading.Thread):
|
||||
application with the result of the process (success or failure).
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('iop 180 run')
|
||||
|
||||
# Show information about the info operation in the Output Tab
|
||||
if self.info_type == 'test_ytdl':
|
||||
|
||||
msg = _(
|
||||
'Starting info operation, testing youtube-dl with specified' \
|
||||
'Starting info operation, testing downloader with specified' \
|
||||
+ ' options',
|
||||
)
|
||||
|
||||
@ -206,7 +196,7 @@ class InfoManager(threading.Thread):
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(1, msg)
|
||||
|
||||
# Convert a path beginning with ~ (not on MS Windows)
|
||||
ytdl_path = self.app_obj.ytdl_path
|
||||
ytdl_path = self.app_obj.check_downloader(self.app_obj.ytdl_path)
|
||||
if os.name != 'nt':
|
||||
ytdl_path = re.sub('^\~', os.path.expanduser('~'), ytdl_path)
|
||||
|
||||
@ -340,7 +330,7 @@ class InfoManager(threading.Thread):
|
||||
# situations)
|
||||
if self.child_process is None:
|
||||
|
||||
msg = _('youtube-dl process did not start')
|
||||
msg = _('System process did not start')
|
||||
self.stderr_list.append(msg)
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
@ -391,9 +381,6 @@ class InfoManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('iop 395 create_child_process')
|
||||
|
||||
info = preexec = None
|
||||
|
||||
if os.name == 'nt':
|
||||
@ -436,9 +423,6 @@ class InfoManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('iop 440 is_child_process_alive')
|
||||
|
||||
if self.child_process is None:
|
||||
return False
|
||||
|
||||
@ -455,9 +439,6 @@ class InfoManager(threading.Thread):
|
||||
Terminates the child process.
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('iop 459 stop_info_operation')
|
||||
|
||||
if self.is_child_process_alive():
|
||||
|
||||
if os.name == 'nt':
|
||||
|
||||
1870
tartube/mainapp.py
1870
tartube/mainapp.py
File diff suppressed because it is too large
Load Diff
1127
tartube/mainwin.py
1127
tartube/mainwin.py
File diff suppressed because it is too large
Load Diff
201
tartube/media.py
201
tartube/media.py
@ -33,6 +33,7 @@ import time
|
||||
|
||||
|
||||
# Import our modules
|
||||
import formats
|
||||
import mainapp
|
||||
import utils
|
||||
# Use same gettext translations
|
||||
@ -1593,10 +1594,12 @@ class Video(GenericMedia):
|
||||
|
||||
"""
|
||||
|
||||
descrip_path = self.get_actual_path_by_ext(app_obj, '.description')
|
||||
text = app_obj.file_manager_obj.load_text(descrip_path)
|
||||
if text is not None:
|
||||
self.set_video_descrip(text, max_length)
|
||||
descrip_path = self.check_actual_path_by_ext(app_obj, '.description')
|
||||
if (descrip_path):
|
||||
|
||||
text = app_obj.file_manager_obj.load_text(descrip_path)
|
||||
if text is not None:
|
||||
self.set_video_descrip(text, max_length)
|
||||
|
||||
|
||||
# Set accessors
|
||||
@ -1846,6 +1849,10 @@ class Video(GenericMedia):
|
||||
|
||||
app_obj (mainapp.TartubeApp): The main application
|
||||
|
||||
Returns:
|
||||
|
||||
The path described above
|
||||
|
||||
"""
|
||||
|
||||
return os.path.abspath(
|
||||
@ -1880,6 +1887,10 @@ class Video(GenericMedia):
|
||||
|
||||
ext (str): The extension, e.g. 'png' or '.png'
|
||||
|
||||
Returns:
|
||||
|
||||
The full file path (the file may or may not exist)
|
||||
|
||||
"""
|
||||
|
||||
# Add the full stop, if not supplied by the calling function
|
||||
@ -1894,6 +1905,129 @@ class Video(GenericMedia):
|
||||
)
|
||||
|
||||
|
||||
def get_actual_path_in_subdirectory_by_ext(self, app_obj, ext):
|
||||
|
||||
"""Can be called by anything.
|
||||
|
||||
Modified version of self.get_actual_path_by_ext().
|
||||
|
||||
The file might be stored in the same directory as its video, or in the
|
||||
sub-directory '.thumbs' (for thumbnails) or '.data' (for everything
|
||||
else).
|
||||
|
||||
self.get_actual_path_by_ext() returns the former; this function returns
|
||||
the latter.
|
||||
|
||||
Args:
|
||||
|
||||
app_obj (mainapp.TartubeApp): The main application
|
||||
|
||||
ext (str): The extension, e.g. 'png' or '.png'
|
||||
|
||||
Returns:
|
||||
|
||||
The full file path (the file may or may not exist)
|
||||
|
||||
"""
|
||||
|
||||
# Add the full stop, if not supplied by the calling function
|
||||
if not ext.find('.') == 0:
|
||||
ext = '.' + ext
|
||||
|
||||
# There are two sub-directories, one for thumbnails, one for metadata
|
||||
if ext in formats.IMAGE_FORMAT_EXT_LIST:
|
||||
|
||||
return os.path.abspath(
|
||||
os.path.join(
|
||||
self.parent_obj.get_actual_dir(app_obj),
|
||||
app_obj.thumbs_sub_dir,
|
||||
self.file_name + ext,
|
||||
),
|
||||
)
|
||||
|
||||
else:
|
||||
|
||||
return os.path.abspath(
|
||||
os.path.join(
|
||||
self.parent_obj.get_actual_dir(app_obj),
|
||||
app_obj.metadata_sub_dir,
|
||||
self.file_name + ext,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def check_actual_path_by_ext(self, app_obj, ext):
|
||||
|
||||
"""Can be called by anything.
|
||||
|
||||
Modified version of self.get_actual_path_by_ext().
|
||||
|
||||
The file has the same name as its video, but with a different extension
|
||||
(for example, the video's thumbnail file).
|
||||
|
||||
The file might be stored in the same directory as its video, or in the
|
||||
sub-directory '.thumbs' (for thumbnails) or '.data' (for everything
|
||||
else).
|
||||
|
||||
This function checks to see whether the file exists in the same
|
||||
directory as its folder and, if so, returns the file path. If not, it
|
||||
checks to see whether the file exists in the '.thumbs' or '.data'
|
||||
sub-directory and, if so, returns the file path.
|
||||
|
||||
Args:
|
||||
|
||||
app_obj (mainapp.TartubeApp): The main application
|
||||
|
||||
ext (str): The extension, e.g. 'png' or '.png'
|
||||
|
||||
Returns:
|
||||
|
||||
The full path to the file if it exists, or None if not
|
||||
|
||||
"""
|
||||
|
||||
# Add the full stop, if not supplied by the calling function
|
||||
if not ext.find('.') == 0:
|
||||
ext = '.' + ext
|
||||
|
||||
# Check the normal location
|
||||
main_path = os.path.abspath(
|
||||
os.path.join(
|
||||
self.parent_obj.get_actual_dir(app_obj),
|
||||
self.file_name + ext,
|
||||
),
|
||||
)
|
||||
|
||||
if os.path.isfile(main_path):
|
||||
return main_path
|
||||
|
||||
# Check the sub-directory location
|
||||
if ext in formats.IMAGE_FORMAT_EXT_LIST:
|
||||
|
||||
subdir_path = os.path.abspath(
|
||||
os.path.join(
|
||||
self.parent_obj.get_actual_dir(app_obj),
|
||||
app_obj.thumbs_sub_dir,
|
||||
self.file_name + ext,
|
||||
),
|
||||
)
|
||||
|
||||
else:
|
||||
|
||||
subdir_path = os.path.abspath(
|
||||
os.path.join(
|
||||
self.parent_obj.get_actual_dir(app_obj),
|
||||
app_obj.metadata_sub_dir,
|
||||
self.file_name + ext,
|
||||
),
|
||||
)
|
||||
|
||||
if os.path.isfile(subdir_path):
|
||||
return subdir_path
|
||||
else:
|
||||
return None
|
||||
|
||||
|
||||
def get_default_path(self, app_obj):
|
||||
|
||||
"""Can be called by anything.
|
||||
@ -1909,6 +2043,10 @@ class Video(GenericMedia):
|
||||
|
||||
app_obj (mainapp.TartubeApp): The main application
|
||||
|
||||
Returns:
|
||||
|
||||
The full file path (the file may or may not exist)
|
||||
|
||||
"""
|
||||
|
||||
return os.path.abspath(
|
||||
@ -1938,6 +2076,10 @@ class Video(GenericMedia):
|
||||
|
||||
ext (str): The extension, e.g. 'png' or '.png'
|
||||
|
||||
Returns:
|
||||
|
||||
The full file path (the file may or may not exist)
|
||||
|
||||
"""
|
||||
|
||||
# Add the full stop, if not supplied by the calling function
|
||||
@ -1952,6 +2094,57 @@ class Video(GenericMedia):
|
||||
)
|
||||
|
||||
|
||||
def get_default_path_in_subdirectory_by_ext(self, app_obj, ext):
|
||||
|
||||
"""Can be called by anything.
|
||||
|
||||
Modified version of self.get_default_path_by_ext().
|
||||
|
||||
The file might be stored in the same directory as its video, or in the
|
||||
sub-directory '.thumbs' (for thumbnails) or '.data' (for everything
|
||||
else).
|
||||
|
||||
self.get_default_path_by_ext() returns the former; this function
|
||||
returns the latter.
|
||||
|
||||
Args:
|
||||
|
||||
app_obj (mainapp.TartubeApp): The main application
|
||||
|
||||
ext (str): The extension, e.g. 'png' or '.png'
|
||||
|
||||
Returns:
|
||||
|
||||
The full file path (the file may or may not exist)
|
||||
|
||||
"""
|
||||
|
||||
# Add the full stop, if not supplied by the calling function
|
||||
if not ext.find('.') == 0:
|
||||
ext = '.' + ext
|
||||
|
||||
# There are two sub-directories, one for thumbnails, one for metadata
|
||||
if ext in formats.IMAGE_FORMAT_EXT_LIST:
|
||||
|
||||
return os.path.abspath(
|
||||
os.path.join(
|
||||
self.parent_obj.get_default_dir(app_obj),
|
||||
app_obj.thumbs_sub_dir,
|
||||
self.file_name + ext,
|
||||
),
|
||||
)
|
||||
|
||||
else:
|
||||
|
||||
return os.path.abspath(
|
||||
os.path.join(
|
||||
self.parent_obj.get_default_dir(app_obj),
|
||||
app_obj.metadata_sub_dir,
|
||||
self.file_name + ext,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def get_file_size_string(self):
|
||||
|
||||
"""Can be called by anything.
|
||||
|
||||
@ -190,10 +190,6 @@ class OptionsManager(object):
|
||||
write_thumbnail (bool): If True youtube-dl will write thumbnail image
|
||||
to disc
|
||||
|
||||
VERBOSITY / SIMULATION OPTIONS
|
||||
|
||||
youtube_dl_debug (bool): When True, will pass '-v' flag to youtube-dl
|
||||
|
||||
WORKAROUNDS
|
||||
|
||||
force_encoding (str): Force the specified encoding
|
||||
@ -338,6 +334,15 @@ class OptionsManager(object):
|
||||
|
||||
TARTUBE OPTIONS (not passed to youtube-dl directly)
|
||||
|
||||
move_description (bool):
|
||||
move_info (bool):
|
||||
move_annotations (bool):
|
||||
move_thumbnail (bool):
|
||||
During a download operation (real or simulated), if these values
|
||||
are True, the video description/JSON/annotations files are moved to
|
||||
a '.data' sub-directory, and the thumbnails are moved to a
|
||||
'.thumbs' sub-directory, inside the directory containing the videos
|
||||
|
||||
keep_description (bool):
|
||||
keep_info (bool):
|
||||
keep_annotations (bool):
|
||||
@ -594,6 +599,10 @@ class OptionsManager(object):
|
||||
'min_filesize_unit' : '',
|
||||
'extra_cmd_string' : '',
|
||||
# TARTUBE OPTIONS
|
||||
'move_description': False,
|
||||
'move_info': False,
|
||||
'move_annotations': False,
|
||||
'move_thumbnail': False,
|
||||
'keep_description': False,
|
||||
'keep_info': False,
|
||||
'keep_annotations': False,
|
||||
@ -624,6 +633,11 @@ class OptionsManager(object):
|
||||
self.options_dict['write_annotations'] = False
|
||||
self.options_dict['write_thumbnail'] = False
|
||||
|
||||
self.options_dict['move_description'] = False
|
||||
self.options_dict['move_info'] = False
|
||||
self.options_dict['move_annotations'] = False
|
||||
self.options_dict['move_thumbnail'] = False
|
||||
|
||||
self.options_dict['keep_description'] = False
|
||||
self.options_dict['keep_info'] = False
|
||||
self.options_dict['keep_annotations'] = False
|
||||
@ -857,6 +871,10 @@ class OptionsParser(object):
|
||||
# OptionHolder('min_filesize_unit', '', ''),
|
||||
# OptionHolder('extra_cmd_string', '', ''),
|
||||
# TARTUBE OPTIONS (not given an options.OptionHolder object)
|
||||
# OptionHolder('move_description', '', False),
|
||||
# OptionHolder('move_info', '', False),
|
||||
# OptionHolder('move_annotations', '', False),
|
||||
# OptionHolder('move_thumbnail', '', False),
|
||||
# OptionHolder('keep_description', '', False),
|
||||
# OptionHolder('keep_info', '', False),
|
||||
# OptionHolder('keep_annotations', '', False),
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
304
tartube/process.py
Normal file
304
tartube/process.py
Normal file
@ -0,0 +1,304 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
# Copyright (C) 2019-2020 A S Lewis
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify it under
|
||||
# the terms of the GNU General Public License as published by the Free Software
|
||||
# Foundation, either version 3 of the License, or (at your option) any later
|
||||
# version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
|
||||
# details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License along with
|
||||
# this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
|
||||
"""Process operation classes."""
|
||||
|
||||
|
||||
# Import Gtk modules
|
||||
import gi
|
||||
from gi.repository import GObject
|
||||
|
||||
|
||||
# Import other modules
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
|
||||
|
||||
# Import our modules
|
||||
import media
|
||||
import utils
|
||||
# Use same gettext translations
|
||||
from mainapp import _
|
||||
|
||||
|
||||
# Classes
|
||||
|
||||
|
||||
class ProcessManager(threading.Thread):
|
||||
|
||||
"""Called by mainapp.TartubeApp.process_manager_start().
|
||||
|
||||
Python class to manage the process operation, in which media.Video objects
|
||||
are sent to FFmpeg for post-processing.
|
||||
|
||||
Args:
|
||||
|
||||
app_obj (mainapp.TartubeApp): The main application
|
||||
|
||||
option_string (str): A string of FFmpeg options (usually a copy of
|
||||
mainapp.TartubeApp.ffmpeg_option_string)
|
||||
|
||||
add_string (str): Text to add to the end of every filename (usually a
|
||||
copy of mainapp.TartubeApp.ffmpeg_add_string)
|
||||
|
||||
regex_string, substitute_string (str): A regex substitution to apply to
|
||||
every filename (usually a copy of
|
||||
mainapp.TartubeApp.ffmpeg_regex_string and
|
||||
.ffmpeg_substitute_string); ignored if regex_string is an empty
|
||||
string, not ignored if substitute_string is an empty string
|
||||
|
||||
ext_string (str): The replacement file extension to use (usually a copy
|
||||
of mainapp.TartubeApp.ffmpeg_ext_string); ignored if an empty
|
||||
string
|
||||
|
||||
delete_flag (bool): True if the old video file should be deleted (and
|
||||
media.Video IVs updated) if FFmpeg's output file has a different
|
||||
name (for example, if the file extension has changed); False
|
||||
otherwise
|
||||
|
||||
video_list (list): A list of media.Video objects to be passed to FFmpeg
|
||||
|
||||
"""
|
||||
|
||||
|
||||
# Standard class methods
|
||||
|
||||
|
||||
def __init__(self, app_obj, option_string, add_string, regex_string,
|
||||
substitute_string, ext_string, delete_flag, video_list):
|
||||
|
||||
super(ProcessManager, self).__init__()
|
||||
|
||||
# IV list - class objects
|
||||
# -----------------------
|
||||
# The mainapp.TartubeApp object
|
||||
self.app_obj = app_obj
|
||||
# A list of media.Video objects to be passed to FFmpeg
|
||||
self.video_list = video_list
|
||||
|
||||
|
||||
# IV list - other
|
||||
# ---------------
|
||||
# Flag set to False if self.stop_process_operation() is called, which
|
||||
# halts the operation immediately
|
||||
self.running_flag = True
|
||||
|
||||
# The time at which the process operation began (in seconds since
|
||||
# epoch)
|
||||
self.start_time = int(time.time())
|
||||
# The time at which the process operation completed (in seconds since
|
||||
# epoch)
|
||||
self.stop_time = None
|
||||
# The time (in seconds) between iterations of the loop in self.run()
|
||||
self.sleep_time = 0.25
|
||||
|
||||
# The number of media.Video objects processed so far...
|
||||
self.job_count = 0
|
||||
# ...and the total number to process (these numbers are displayed in
|
||||
# the progress bar in the Videos tab)
|
||||
self.job_total = len(video_list)
|
||||
|
||||
# A string of FFmpeg options (usually a copy of
|
||||
# mainapp.TartubeApp.ffmpeg_option_string)
|
||||
self.option_string = option_string
|
||||
# That string, converted to a list of options by self.run
|
||||
self.option_list = []
|
||||
|
||||
# Text to add to the end of every filename (usually a copy of
|
||||
# mainapp.TartubeApp.ffmpeg_add_string)
|
||||
self.add_string = add_string
|
||||
# A regex substitution to apply to every filename (usually a copy of
|
||||
# mainapp.TartubeApp.ffmpeg_regex_string and
|
||||
# .ffmpeg_substitute_string); ignored if regex_string is an
|
||||
# empty string, not ignored if substitute_string is an empty string
|
||||
self.regex_string = regex_string
|
||||
self.substitute_string = substitute_string
|
||||
# The replacement file extension to use (usually a copy of
|
||||
# mainapp.TartubeApp.ffmpeg_ext_string); ignored if an empty string
|
||||
self.ext_string = ext_string
|
||||
# Flag set to True if the old video file should be deleted (and
|
||||
# media.Video IVs updated) if FFmpeg's output file has a different
|
||||
# name (for example, if the file extension has changed); False
|
||||
# otherwise
|
||||
self.delete_flag = delete_flag
|
||||
|
||||
# Code
|
||||
# ----
|
||||
|
||||
# Prepare a list of FFmpeg options, from the option string specified by
|
||||
# the user
|
||||
self.option_list = utils.parse_ytdl_options(self.option_string)
|
||||
|
||||
# Let's get this party started!
|
||||
self.start()
|
||||
|
||||
|
||||
# Public class methods
|
||||
|
||||
|
||||
def run(self):
|
||||
|
||||
"""Called as a result of self.__init__().
|
||||
|
||||
Calls FFmpegManager.run_ffmpeg for every media.Video object in the
|
||||
list.
|
||||
|
||||
Then informs the main application that the process operation is
|
||||
complete.
|
||||
"""
|
||||
|
||||
# Show information about the process operation in the Output Tab
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
_('Starting process operation'),
|
||||
)
|
||||
|
||||
# Process each video in turn
|
||||
while self.running_flag and self.video_list:
|
||||
|
||||
self.process_video(self.video_list.pop(0))
|
||||
|
||||
# Pause a moment, before the next iteration of the loop (don't want
|
||||
# to hog resources)
|
||||
time.sleep(self.sleep_time)
|
||||
|
||||
# Operation complete. Set the stop time
|
||||
self.stop_time = int(time.time())
|
||||
|
||||
# Show a confirmation in the Output Tab
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
_('Process operation finished'),
|
||||
)
|
||||
|
||||
# Let the timer run for a few more seconds to prevent Gtk errors (for
|
||||
# systems with Gtk < 3.24)
|
||||
GObject.timeout_add(
|
||||
0,
|
||||
self.app_obj.process_manager_halt_timer,
|
||||
)
|
||||
|
||||
|
||||
def process_video(self, video_obj):
|
||||
|
||||
"""Called by self.run().
|
||||
|
||||
Sends a single video to FFmpeg for post-processing.
|
||||
|
||||
Args:
|
||||
|
||||
video_obj (media.Video): The video to be sent to FFmpeg
|
||||
|
||||
"""
|
||||
|
||||
# Get the path to the video file, which might be in the directory of
|
||||
# its parent channel/playlist/folder, or in a different directory
|
||||
# altogether
|
||||
input_path = video_obj.get_actual_path(self.app_obj)
|
||||
# Set the output path; the same as the input path, unless the user has
|
||||
# requested changes
|
||||
output_file, output_ext = os.path.splitext(input_path)
|
||||
|
||||
if self.add_string != '':
|
||||
output_file += self.add_string
|
||||
|
||||
if self.substitute_string != '':
|
||||
output_file = re.sub(
|
||||
self.regex_string,
|
||||
self.substitute_string,
|
||||
output_file,
|
||||
)
|
||||
|
||||
if self.ext_string != '':
|
||||
output_ext = self.ext_string
|
||||
|
||||
output_path = output_file + output_ext
|
||||
|
||||
# Update the main window's progress bar
|
||||
self.job_count += 1
|
||||
GObject.timeout_add(
|
||||
0,
|
||||
self.app_obj.main_win_obj.update_progress_bar,
|
||||
video_obj.name,
|
||||
self.job_count,
|
||||
self.job_total,
|
||||
)
|
||||
|
||||
# Update our progress in the Output Tab
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Video') + ' ' + str(self.job_count) + '/' \
|
||||
+ str(self.job_total) + ': ' + video_obj.name,
|
||||
)
|
||||
|
||||
# Show the system command we're about to execute...
|
||||
test_list = self.app_obj.ffmpeg_manager_obj.run_ffmpeg(
|
||||
input_path,
|
||||
output_path,
|
||||
self.option_list,
|
||||
True,
|
||||
)
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Input:') + ' ' + ' '.join(test_list[1]),
|
||||
)
|
||||
|
||||
# ...and then send the command to FFmpeg for processing, which returns
|
||||
# a list in the form (success_flag, optional_message)
|
||||
result_list = self.app_obj.ffmpeg_manager_obj.run_ffmpeg(
|
||||
input_path,
|
||||
output_path,
|
||||
self.option_list,
|
||||
)
|
||||
|
||||
if not result_list or not result_list[0]:
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Output: FAILED:') + ' ' + result_list[1],
|
||||
)
|
||||
|
||||
else:
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Output:') + ' ' + output_path,
|
||||
)
|
||||
|
||||
# Delete the original video file, and update media.Video IVs, if
|
||||
# required
|
||||
if self.delete_flag \
|
||||
and os.path.isfile(input_path) \
|
||||
and os.path.isfile(output_path) \
|
||||
and input_path != output_path:
|
||||
os.remove(input_path)
|
||||
video_obj.set_file(output_file, output_ext)
|
||||
|
||||
|
||||
def stop_process_operation(self):
|
||||
|
||||
"""Called by mainapp.TartubeApp.do_shutdown(), .stop_continue(),
|
||||
.on_button_stop_operation() and mainwin.MainWin.on_stop_menu_item().
|
||||
|
||||
Stops the process operation.
|
||||
"""
|
||||
|
||||
self.running_flag = False
|
||||
@ -39,10 +39,6 @@ import utils
|
||||
from mainapp import _
|
||||
|
||||
|
||||
# Debugging flag (calls utils.debug_time at the start of every function)
|
||||
DEBUG_FUNC_FLAG = False
|
||||
|
||||
|
||||
# Classes
|
||||
|
||||
|
||||
@ -69,9 +65,6 @@ class RefreshManager(threading.Thread):
|
||||
|
||||
def __init__(self, app_obj, init_obj=None):
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('rop 73 __init__')
|
||||
|
||||
super(RefreshManager, self).__init__()
|
||||
|
||||
# IV list - class objects
|
||||
@ -139,9 +132,6 @@ class RefreshManager(threading.Thread):
|
||||
complete.
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('rop 143 run')
|
||||
|
||||
# Show information about the refresh operation in the Output Tab
|
||||
if not self.init_obj:
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
@ -248,9 +238,6 @@ class RefreshManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('rop 252 refresh_from_default_destination')
|
||||
|
||||
# Update the main window's progress bar
|
||||
self.job_count += 1
|
||||
GObject.timeout_add(
|
||||
@ -519,9 +506,6 @@ class RefreshManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('rop 519 refresh_from_actual_destination')
|
||||
|
||||
# Update the main window's progress bar
|
||||
self.job_count += 1
|
||||
GObject.timeout_add(
|
||||
@ -617,7 +601,4 @@ class RefreshManager(threading.Thread):
|
||||
Stops the refresh operation.
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('rop 613 stop_refresh_operation')
|
||||
|
||||
self.running_flag = False
|
||||
|
||||
@ -42,8 +42,8 @@ import mainapp
|
||||
|
||||
# 'Global' variables
|
||||
__packagename__ = 'tartube'
|
||||
__version__ = '2.1.080'
|
||||
__date__ = '13 Aug 2020'
|
||||
__version__ = '2.2.0'
|
||||
__date__ = '30 Sep 2020'
|
||||
__copyright__ = 'Copyright \xa9 2019-2020 A S Lewis'
|
||||
__license__ = """
|
||||
Copyright \xa9 2019-2020 A S Lewis.
|
||||
|
||||
@ -29,7 +29,7 @@
|
||||
|
||||
|
||||
# Import our modules
|
||||
# ...
|
||||
import media
|
||||
|
||||
|
||||
# Functions
|
||||
@ -39,11 +39,11 @@ def add_test_media(app_obj):
|
||||
|
||||
"""Called by mainapp.TartubeApp.on_menu_test().
|
||||
|
||||
Add a set of media data objects for testing. This function can only be
|
||||
Adds a set of media data objects for testing. This function can only be
|
||||
called if the debugging flags are set.
|
||||
|
||||
Enable/disable various media objects by changing the 0s and 1s in the code
|
||||
below.
|
||||
Enables/disables various media objects by changing the 0s and 1s in the
|
||||
code below.
|
||||
|
||||
The videos, channels and playlists listed here have been chosen because
|
||||
they are short. They have no connection to the Tartube developers.
|
||||
@ -135,3 +135,108 @@ def add_test_media(app_obj):
|
||||
)
|
||||
app_obj.main_win_obj.video_index_add_row(folder4)
|
||||
|
||||
|
||||
def run_test_code(app_obj):
|
||||
|
||||
"""Called by mainapp.TartubeApp.on_menu_test_code().
|
||||
|
||||
Executes some arbitrary test code, and returns a result.
|
||||
|
||||
Args:
|
||||
|
||||
app_obj (mainapp.TartubeApp): The main application
|
||||
|
||||
"""
|
||||
|
||||
# ... insert code here ...
|
||||
|
||||
# ...
|
||||
|
||||
# ... insert code here ...
|
||||
|
||||
return "Hello world"
|
||||
|
||||
|
||||
def setup_screenshots(app_obj):
|
||||
|
||||
"""Call this function from testing.run_test_code, when required.
|
||||
|
||||
Sets up four fake channels, with fake videos, in order to take screenshots
|
||||
for the README.
|
||||
|
||||
"""
|
||||
|
||||
folder = app_obj.add_folder(
|
||||
'Comedy',
|
||||
None, # No parent
|
||||
)
|
||||
app_obj.main_win_obj.video_index_add_row(folder)
|
||||
|
||||
folder2 = app_obj.add_folder(
|
||||
'Music',
|
||||
None, # No parent
|
||||
)
|
||||
app_obj.main_win_obj.video_index_add_row(folder2)
|
||||
|
||||
|
||||
channel_list = [
|
||||
'PewDiePie',
|
||||
'https://www.youtube.com/user/PewDiePie/videos',
|
||||
4205,
|
||||
'Comedy',
|
||||
'T-Series',
|
||||
'https://www.youtube.com/aashiqui2/videos',
|
||||
14705,
|
||||
'Music',
|
||||
'The Beatles',
|
||||
'https://www.youtube.com/c/TheBeatles/videos',
|
||||
219,
|
||||
'Music',
|
||||
'Luke TheNotable',
|
||||
'https://www.youtube.com/c/LukeTheNotable/videos',
|
||||
# 486,
|
||||
476,
|
||||
'Comedy',
|
||||
]
|
||||
|
||||
while channel_list:
|
||||
|
||||
channel_name = channel_list.pop(0)
|
||||
channel_url = channel_list.pop(0)
|
||||
video_count = channel_list.pop(0)
|
||||
folder_name = channel_list.pop(0)
|
||||
folder_dbid = app_obj.media_name_dict[folder_name]
|
||||
folder_obj = app_obj.media_reg_dict[folder_dbid]
|
||||
|
||||
channel_obj = app_obj.add_channel(
|
||||
channel_name,
|
||||
folder_obj,
|
||||
channel_url,
|
||||
None,
|
||||
)
|
||||
app_obj.main_win_obj.video_index_add_row(channel_obj)
|
||||
|
||||
for i in range(0, video_count):
|
||||
video_obj = app_obj.add_video(
|
||||
channel_obj,
|
||||
'https://www.youtube.com/', # Fake URL
|
||||
)
|
||||
video_obj.name = video_obj.nickname = 'Fake video'
|
||||
video_obj.upload_time = 1
|
||||
video_obj.receive_time = 1
|
||||
|
||||
app_obj.mark_video_downloaded(video_obj, True)
|
||||
|
||||
|
||||
def setup_screenshots2(app_obj):
|
||||
|
||||
"""Call this function from testing.run_test_code, when required.
|
||||
|
||||
Makes sure all videos are marked as downloaded.
|
||||
"""
|
||||
|
||||
for media_data_obj in app_obj.media_reg_dict.values():
|
||||
if isinstance(media_data_obj, media.Video) \
|
||||
and not media_data_obj.dl_flag:
|
||||
app_obj.mark_video_downloaded(media_data_obj, True)
|
||||
|
||||
|
||||
680
tartube/tidy.py
680
tartube/tidy.py
@ -33,6 +33,7 @@ except:
|
||||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import threading
|
||||
import time
|
||||
|
||||
@ -45,10 +46,6 @@ import utils
|
||||
from mainapp import _
|
||||
|
||||
|
||||
# Debugging flag (calls utils.debug_time at the start of every function)
|
||||
DEBUG_FUNC_FLAG = False
|
||||
|
||||
|
||||
# Classes
|
||||
|
||||
|
||||
@ -86,20 +83,26 @@ class TidyManager(threading.Thread):
|
||||
should be deleted (as artefacts of post-processing with FFmpeg
|
||||
or AVConv)
|
||||
|
||||
del_archive_flag: True if all youtube-dl archive files should be
|
||||
deleted
|
||||
|
||||
move_thumb_flag: True if all thumbnail files should be moved into a
|
||||
subdirectory
|
||||
|
||||
del_thumb_flag: True if all thumbnail files should be deleted
|
||||
|
||||
convert_webp_flag: True if all .webp thumbnail files should be
|
||||
converted to .jpg
|
||||
|
||||
move_data_flag: True if description, metadata (JSON) and annotation
|
||||
files should be moved into a subdirectory
|
||||
|
||||
del_descrip_flag: True if all description files should be deleted
|
||||
|
||||
del_json_flag: True if all metadata (JSON) files should be deleted
|
||||
|
||||
del_xml_flag: True if all annotation files should be deleted
|
||||
|
||||
del_thumb_flag: True if all thumbnail files should be deleted
|
||||
|
||||
del_webp_flag: True if all thumbnail files in .webp or malformed
|
||||
.jpg format should be deleted (see comments below)
|
||||
|
||||
del_archive_flag: True if all youtube-dl archive files should be
|
||||
deleted
|
||||
|
||||
"""
|
||||
|
||||
|
||||
@ -108,9 +111,6 @@ class TidyManager(threading.Thread):
|
||||
|
||||
def __init__(self, app_obj, choices_dict):
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 112 __init__')
|
||||
|
||||
super(TidyManager, self).__init__()
|
||||
|
||||
# IV list - class objects
|
||||
@ -151,23 +151,24 @@ class TidyManager(threading.Thread):
|
||||
# True if all video/audio files with the same name should be deleted
|
||||
# (as artefacts of post-processing with FFmpeg or AVConv)
|
||||
self.del_others_flag = choices_dict['del_others_flag']
|
||||
# True if all youtube-dl archive files should be deleted
|
||||
self.del_archive_flag = choices_dict['del_archive_flag']
|
||||
# True if all thumbnail files should be moved into a subdirectory
|
||||
self.move_thumb_flag = choices_dict['move_thumb_flag']
|
||||
# True if all thumbnail files should be deleted
|
||||
self.del_thumb_flag = choices_dict['del_thumb_flag']
|
||||
# True if all .webp thumbnail files should be converted to .jpg.
|
||||
# Requires mainapp.TartubeApp.ffmpeg_fail_flag set to False
|
||||
self.convert_webp_flag = choices_dict['convert_webp_flag']
|
||||
# True if description, metadata (JSON) and annotation files should be
|
||||
# moved into a subdirectory
|
||||
self.move_data_flag = choices_dict['move_data_flag']
|
||||
# True if all description files should be deleted
|
||||
self.del_descrip_flag = choices_dict['del_descrip_flag']
|
||||
# True if all metadata (JSON) files should be deleted
|
||||
self.del_json_flag = choices_dict['del_json_flag']
|
||||
# True if all annotation files should be deleted
|
||||
self.del_xml_flag = choices_dict['del_xml_flag']
|
||||
# True if all thumbnail files should be deleted
|
||||
self.del_thumb_flag = choices_dict['del_thumb_flag']
|
||||
# v2.1.027. In June 2020, YouTube started serving .webp thumbnails.
|
||||
# At the time of writing, Gtk can't display them. A youtube-dl fix is
|
||||
# expected, which will convert .webp thumbnails to .jpg; in
|
||||
# anticipation of that, we add an option to remove .webp files
|
||||
# True if all thumbnail files in .webp or malformed .jpg format should
|
||||
# be deleted
|
||||
self.del_webp_flag = choices_dict['del_webp_flag']
|
||||
# True if all youtube-dl archive files should be deleted
|
||||
self.del_archive_flag = choices_dict['del_archive_flag']
|
||||
|
||||
# The number of media data objects whose directories have been tidied
|
||||
# so far...
|
||||
@ -183,17 +184,23 @@ class TidyManager(threading.Thread):
|
||||
self.video_no_exist_count = 0
|
||||
self.video_deleted_count = 0
|
||||
self.other_deleted_count = 0
|
||||
self.archive_deleted_count = 0
|
||||
self.thumb_moved_count = 0
|
||||
self.thumb_deleted_count = 0
|
||||
self.webp_converted_count = 0
|
||||
self.data_moved_count = 0
|
||||
self.descrip_deleted_count = 0
|
||||
self.json_deleted_count = 0
|
||||
self.xml_deleted_count = 0
|
||||
self.thumb_deleted_count = 0
|
||||
self.webp_deleted_count = 0
|
||||
self.archive_deleted_count = 0
|
||||
|
||||
|
||||
# Code
|
||||
# ----
|
||||
|
||||
# Do not convert .webp thumbnails, if not allowed
|
||||
if self.app_obj.ffmpeg_fail_flag:
|
||||
self.convert_webp_flag = False
|
||||
|
||||
# Let's get this party started!
|
||||
self.start()
|
||||
|
||||
@ -216,9 +223,6 @@ class TidyManager(threading.Thread):
|
||||
complete.
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 220 run')
|
||||
|
||||
# Show information about the tidy operation in the Output Tab
|
||||
if not self.init_obj:
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
@ -291,6 +295,57 @@ class TidyManager(threading.Thread):
|
||||
' ' + _('Delete other video/audio files:') + ' ' + text,
|
||||
)
|
||||
|
||||
if self.del_archive_flag:
|
||||
text = _('YES')
|
||||
else:
|
||||
text = _('NO')
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Delete downloader archive files:') + ' ' + text,
|
||||
)
|
||||
|
||||
if self.move_thumb_flag:
|
||||
text = _('YES')
|
||||
else:
|
||||
text = _('NO')
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Move thumbnails into own folder:') + ' ' + text,
|
||||
)
|
||||
|
||||
if self.del_thumb_flag:
|
||||
text = _('YES')
|
||||
else:
|
||||
text = _('NO')
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Delete all thumbnail files:') + ' ' + text,
|
||||
)
|
||||
|
||||
if self.convert_webp_flag:
|
||||
text = _('YES')
|
||||
else:
|
||||
text = _('NO')
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Convert .webp thumbnails to .jpg:') + ' ' + text,
|
||||
)
|
||||
|
||||
if self.move_data_flag:
|
||||
text = _('YES')
|
||||
else:
|
||||
text = _('NO')
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Move other metadata files into own folder:') \
|
||||
+ ' ' + text,
|
||||
)
|
||||
|
||||
if self.del_descrip_flag:
|
||||
text = _('YES')
|
||||
else:
|
||||
@ -321,36 +376,6 @@ class TidyManager(threading.Thread):
|
||||
' ' + _('Delete all annotation files:') + ' ' + text,
|
||||
)
|
||||
|
||||
if self.del_thumb_flag:
|
||||
text = _('YES')
|
||||
else:
|
||||
text = _('NO')
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Delete all thumbnail files:') + ' ' + text,
|
||||
)
|
||||
|
||||
if self.del_webp_flag:
|
||||
text = _('YES')
|
||||
else:
|
||||
text = _('NO')
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Delete .webp/malformed .jpg files:') + ' ' + text,
|
||||
)
|
||||
|
||||
if self.del_archive_flag:
|
||||
text = _('YES')
|
||||
else:
|
||||
text = _('NO')
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Delete youtube-dl archive files:') + ' ' + text,
|
||||
)
|
||||
|
||||
# Compile a list of channels, playlists and folders to tidy up (each
|
||||
# one has their own sub-directory inside Tartube's data directory)
|
||||
obj_list = []
|
||||
@ -429,6 +454,46 @@ class TidyManager(threading.Thread):
|
||||
+ str(self.other_deleted_count),
|
||||
)
|
||||
|
||||
if self.del_archive_flag:
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Downloader archive files deleted:') + ' ' \
|
||||
+ str(self.archive_deleted_count),
|
||||
)
|
||||
|
||||
if self.move_thumb_flag:
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Thumbnail files moved:') + ' ' \
|
||||
+ str(self.thumb_moved_count),
|
||||
)
|
||||
|
||||
if self.del_thumb_flag:
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Thumbnail files deleted:') + ' ' \
|
||||
+ str(self.thumb_deleted_count),
|
||||
)
|
||||
|
||||
if self.convert_webp_flag:
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('.webp thumbnails converted to .jpg:') + ' ' \
|
||||
+ str(self.webp_converted_count),
|
||||
)
|
||||
|
||||
if self.move_data_flag:
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Other metadata files moved:') + ' ' \
|
||||
+ str(self.data_moved_count),
|
||||
)
|
||||
|
||||
if self.del_descrip_flag:
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
@ -453,30 +518,6 @@ class TidyManager(threading.Thread):
|
||||
+ str(self.xml_deleted_count),
|
||||
)
|
||||
|
||||
if self.del_thumb_flag:
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('Thumbnail files deleted:') + ' ' \
|
||||
+ str(self.thumb_deleted_count),
|
||||
)
|
||||
|
||||
if self.del_webp_flag:
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('.webp/malformed .jpg files deleted:') + ' ' \
|
||||
+ str(self.webp_deleted_count),
|
||||
)
|
||||
|
||||
if self.del_archive_flag:
|
||||
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
' ' + _('youtube-dl archive files deleted:') + ' ' \
|
||||
+ str(self.archive_deleted_count),
|
||||
)
|
||||
|
||||
# Let the timer run for a few more seconds to prevent Gtk errors (for
|
||||
# systems with Gtk < 3.24)
|
||||
GObject.timeout_add(
|
||||
@ -498,9 +539,6 @@ class TidyManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 502 tidy_directory')
|
||||
|
||||
# Update the main window's progress bar
|
||||
self.job_count += 1
|
||||
GObject.timeout_add(
|
||||
@ -527,6 +565,21 @@ class TidyManager(threading.Thread):
|
||||
if self.del_video_flag:
|
||||
self.delete_video(media_data_obj)
|
||||
|
||||
if self.del_archive_flag:
|
||||
self.delete_archive(media_data_obj)
|
||||
|
||||
if self.move_thumb_flag:
|
||||
self.move_thumb(media_data_obj)
|
||||
|
||||
if self.del_thumb_flag:
|
||||
self.delete_thumb(media_data_obj)
|
||||
|
||||
if self.convert_webp_flag:
|
||||
self.convert_webp(media_data_obj)
|
||||
|
||||
if self.move_data_flag:
|
||||
self.move_data(media_data_obj)
|
||||
|
||||
if self.del_descrip_flag:
|
||||
self.delete_descrip(media_data_obj)
|
||||
|
||||
@ -536,15 +589,6 @@ class TidyManager(threading.Thread):
|
||||
if self.del_xml_flag:
|
||||
self.delete_xml(media_data_obj)
|
||||
|
||||
if self.del_thumb_flag:
|
||||
self.delete_thumb(media_data_obj)
|
||||
|
||||
if self.del_webp_flag:
|
||||
self.delete_webp(media_data_obj)
|
||||
|
||||
if self.del_archive_flag:
|
||||
self.delete_archive(media_data_obj)
|
||||
|
||||
|
||||
def check_video_corrupt(self, media_data_obj):
|
||||
|
||||
@ -560,9 +604,6 @@ class TidyManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 564 check_video_corrupt')
|
||||
|
||||
for video_obj in media_data_obj.compile_all_videos( [] ):
|
||||
|
||||
if video_obj.file_name is not None \
|
||||
@ -638,9 +679,6 @@ class TidyManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 642 check_videos_exist')
|
||||
|
||||
for video_obj in media_data_obj.compile_all_videos( [] ):
|
||||
|
||||
if video_obj.file_name is not None:
|
||||
@ -699,9 +737,6 @@ class TidyManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 703 delete_video')
|
||||
|
||||
ext_list = formats.VIDEO_FORMAT_LIST.copy()
|
||||
ext_list.extend(formats.AUDIO_FORMAT_LIST)
|
||||
|
||||
@ -784,12 +819,12 @@ class TidyManager(threading.Thread):
|
||||
self.other_deleted_count += 1
|
||||
|
||||
|
||||
def delete_descrip(self, media_data_obj):
|
||||
def delete_archive(self, media_data_obj):
|
||||
|
||||
"""Called by self.tidy_directory().
|
||||
|
||||
Checks all child videos of the specified media data object. If the
|
||||
associated description file exists, delete it.
|
||||
Checks the specified media data object's directory. If a youtube-dl
|
||||
archive file is found there, delete it.
|
||||
|
||||
Args:
|
||||
|
||||
@ -798,43 +833,26 @@ class TidyManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 802 delete_descrip')
|
||||
archive_path = os.path.abspath(
|
||||
os.path.join(
|
||||
media_data_obj.get_default_dir(self.app_obj),
|
||||
'ytdl-archive.txt',
|
||||
),
|
||||
)
|
||||
|
||||
for video_obj in media_data_obj.compile_all_videos( [] ):
|
||||
if os.path.isfile(archive_path):
|
||||
|
||||
if video_obj.file_name is not None:
|
||||
|
||||
descrip_path = video_obj.get_actual_path_by_ext(
|
||||
self.app_obj,
|
||||
'.description',
|
||||
)
|
||||
|
||||
# If the video's parent container has an alternative download
|
||||
# destination set, we must check the corresponding media
|
||||
# data object. If the latter also has a media.Video object
|
||||
# matching this video, then this function returns None and
|
||||
# nothing is deleted
|
||||
descrip_path = self.check_video_in_actual_dir(
|
||||
media_data_obj,
|
||||
video_obj,
|
||||
descrip_path,
|
||||
)
|
||||
|
||||
if descrip_path is not None \
|
||||
and os.path.isfile(descrip_path):
|
||||
|
||||
# Delete the description file
|
||||
os.remove(descrip_path)
|
||||
self.descrip_deleted_count += 1
|
||||
# Delete the archive file
|
||||
os.remove(archive_path)
|
||||
self.archive_deleted_count += 1
|
||||
|
||||
|
||||
def delete_json(self, media_data_obj):
|
||||
def move_thumb(self, media_data_obj):
|
||||
|
||||
"""Called by self.tidy_directory().
|
||||
|
||||
Checks all child videos of the specified media data object. If the
|
||||
associated metadata (JSON) file exists, delete it.
|
||||
associated thumbnail file exists, moves it into its own sub-directory.
|
||||
|
||||
Args:
|
||||
|
||||
@ -843,80 +861,53 @@ class TidyManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 847 delete_json')
|
||||
|
||||
for video_obj in media_data_obj.compile_all_videos( [] ):
|
||||
|
||||
if video_obj.file_name is not None:
|
||||
|
||||
json_path = video_obj.get_actual_path_by_ext(
|
||||
# Thumbnails might be in one of four locations. If the
|
||||
# thumbnail has already been moved into /.thumbs, then of
|
||||
# course we don't move it again (and this function returns an
|
||||
# empty list)
|
||||
path_list = utils.find_thumbnail_restricted(
|
||||
self.app_obj,
|
||||
'.info.json',
|
||||
)
|
||||
|
||||
# If the video's parent container has an alternative download
|
||||
# destination set, we must check the corresponding media
|
||||
# data object. If the latter also has a media.Video object
|
||||
# matching this video, then this function returns None and
|
||||
# nothing is deleted
|
||||
json_path = self.check_video_in_actual_dir(
|
||||
media_data_obj,
|
||||
video_obj,
|
||||
json_path,
|
||||
)
|
||||
|
||||
if json_path is not None \
|
||||
and os.path.isfile(json_path):
|
||||
if path_list:
|
||||
|
||||
# Delete the metadata file
|
||||
os.remove(json_path)
|
||||
self.json_deleted_count += 1
|
||||
main_path = os.path.abspath(
|
||||
os.path.join(
|
||||
path_list[0], path_list[1],
|
||||
),
|
||||
)
|
||||
|
||||
subdir = os.path.abspath(
|
||||
os.path.join(
|
||||
path_list[0], self.app_obj.thumbs_sub_dir,
|
||||
),
|
||||
)
|
||||
|
||||
def delete_xml(self, media_data_obj):
|
||||
subdir_path = os.path.abspath(
|
||||
os.path.join(
|
||||
path_list[0],
|
||||
self.app_obj.thumbs_sub_dir,
|
||||
path_list[1],
|
||||
),
|
||||
)
|
||||
|
||||
"""Called by self.tidy_directory().
|
||||
if os.path.isfile(main_path) \
|
||||
and not os.path.isfile(subdir_path):
|
||||
|
||||
Checks all child videos of the specified media data object. If the
|
||||
associated annotation file exists, delete it.
|
||||
try:
|
||||
if not os.path.isdir(subdir):
|
||||
os.makedirs(subdir)
|
||||
|
||||
Args:
|
||||
shutil.move(main_path, subdir_path)
|
||||
self.thumb_moved_count += 1
|
||||
|
||||
media_data_obj (media.Channel, media.Playlist or media.Folder):
|
||||
The media data object whose directory must be tidied up
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 892 delete_xml')
|
||||
|
||||
for video_obj in media_data_obj.compile_all_videos( [] ):
|
||||
|
||||
if video_obj.file_name is not None:
|
||||
|
||||
xml_path = video_obj.get_actual_path_by_ext(
|
||||
self.app_obj,
|
||||
'.annotations.xml',
|
||||
)
|
||||
|
||||
# If the video's parent container has an alternative download
|
||||
# destination set, we must check the corresponding media
|
||||
# data object. If the latter also has a media.Video object
|
||||
# matching this video, then this function returns None and
|
||||
# nothing is deleted
|
||||
xml_path = self.check_video_in_actual_dir(
|
||||
media_data_obj,
|
||||
video_obj,
|
||||
xml_path,
|
||||
)
|
||||
|
||||
if xml_path is not None \
|
||||
and os.path.isfile(xml_path):
|
||||
|
||||
# Delete the annotation file
|
||||
os.remove(xml_path)
|
||||
self.xml_deleted_count += 1
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
def delete_thumb(self, media_data_obj):
|
||||
@ -933,14 +924,11 @@ class TidyManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 937 delete_thumb')
|
||||
|
||||
for video_obj in media_data_obj.compile_all_videos( [] ):
|
||||
|
||||
if video_obj.file_name is not None:
|
||||
|
||||
# Thumbnails might be in one of two locations
|
||||
# Thumbnails might be in one of four locations
|
||||
thumb_path = utils.find_thumbnail(self.app_obj, video_obj)
|
||||
|
||||
# If the video's parent container has an alternative download
|
||||
@ -964,13 +952,13 @@ class TidyManager(threading.Thread):
|
||||
self.thumb_deleted_count += 1
|
||||
|
||||
|
||||
def delete_webp(self, media_data_obj):
|
||||
def convert_webp(self, media_data_obj):
|
||||
|
||||
"""Called by self.tidy_directory().
|
||||
|
||||
Checks all child videos of the specified media data object. If the
|
||||
associated thumbnail file in a .webp or malformed .jpg format exists,
|
||||
delete it
|
||||
convert it to .jpg.
|
||||
|
||||
Args:
|
||||
|
||||
@ -979,14 +967,11 @@ class TidyManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 983 delete_webp')
|
||||
|
||||
for video_obj in media_data_obj.compile_all_videos( [] ):
|
||||
|
||||
if video_obj.file_name is not None:
|
||||
|
||||
# Thumbnails might be in one of two locations
|
||||
# Thumbnails might be in one of four locations
|
||||
thumb_path = utils.find_thumbnail_webp(self.app_obj, video_obj)
|
||||
|
||||
# If the video's parent container has an alternative download
|
||||
@ -1005,17 +990,26 @@ class TidyManager(threading.Thread):
|
||||
if thumb_path is not None \
|
||||
and os.path.isfile(thumb_path):
|
||||
|
||||
# Delete the thumbnail file
|
||||
os.remove(thumb_path)
|
||||
self.webp_deleted_count += 1
|
||||
# Convert to .jpg
|
||||
if not self.app_obj.ffmpeg_manager_obj.convert_webp(
|
||||
thumb_path
|
||||
):
|
||||
# FFmpeg is probably not installed; don't try any more
|
||||
# conversions
|
||||
self.convert_webp_flag = False
|
||||
self.app_obj.set_ffmpeg_fail_flag(True)
|
||||
|
||||
else:
|
||||
|
||||
self.webp_converted_count += 1
|
||||
|
||||
|
||||
def delete_archive(self, media_data_obj):
|
||||
def move_data(self, media_data_obj):
|
||||
|
||||
"""Called by self.tidy_directory().
|
||||
|
||||
Checks the specified media data object's directory. If a youtube-dl
|
||||
archive file is found there, delete it.
|
||||
Checks all child videos of the specified media data object. If the
|
||||
associated thumbnail file exists, moves it into its own sub-directory.
|
||||
|
||||
Args:
|
||||
|
||||
@ -1024,21 +1018,230 @@ class TidyManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 1028 delete_archive')
|
||||
for video_obj in media_data_obj.compile_all_videos( [] ):
|
||||
|
||||
archive_path = os.path.abspath(
|
||||
os.path.join(
|
||||
media_data_obj.get_default_dir(self.app_obj),
|
||||
'ytdl-archive.txt',
|
||||
),
|
||||
)
|
||||
if video_obj.file_name is not None:
|
||||
|
||||
if os.path.isfile(archive_path):
|
||||
# Description/JSON/annotations files might be in one of four
|
||||
# locations. If the file has already been moved into /.data,
|
||||
# then of course we don't move it again
|
||||
for ext in ['.description', '.info.json', '.annotations.xml']:
|
||||
|
||||
# Delete the archive file
|
||||
os.remove(archive_path)
|
||||
self.archive_deleted_count += 1
|
||||
main_path = video_obj.get_actual_path_by_ext(
|
||||
self.app_obj,
|
||||
ext,
|
||||
)
|
||||
|
||||
subdir = os.path.abspath(
|
||||
os.path.join(
|
||||
video_obj.parent_obj.get_actual_dir(self.app_obj),
|
||||
self.app_obj.metadata_sub_dir,
|
||||
),
|
||||
)
|
||||
|
||||
subdir_path \
|
||||
= video_obj.get_actual_path_in_subdirectory_by_ext(
|
||||
self.app_obj,
|
||||
ext,
|
||||
)
|
||||
|
||||
if os.path.isfile(main_path) \
|
||||
and not os.path.isfile(subdir_path):
|
||||
|
||||
try:
|
||||
if not os.path.isdir(subdir):
|
||||
os.makedirs(subdir)
|
||||
|
||||
# (os.rename sometimes fails on external hard
|
||||
# drives; this is safer)
|
||||
shutil.move(main_path, subdir_path)
|
||||
self.data_moved_count += 1
|
||||
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
def delete_descrip(self, media_data_obj):
|
||||
|
||||
"""Called by self.tidy_directory().
|
||||
|
||||
Checks all child videos of the specified media data object. If the
|
||||
associated description file exists, delete it.
|
||||
|
||||
Args:
|
||||
|
||||
media_data_obj (media.Channel, media.Playlist or media.Folder):
|
||||
The media data object whose directory must be tidied up
|
||||
|
||||
"""
|
||||
|
||||
for video_obj in media_data_obj.compile_all_videos( [] ):
|
||||
|
||||
if video_obj.file_name is not None:
|
||||
|
||||
main_path = video_obj.get_actual_path_by_ext(
|
||||
self.app_obj,
|
||||
'.description',
|
||||
)
|
||||
|
||||
# If the video's parent container has an alternative download
|
||||
# destination set, we must check the corresponding media
|
||||
# data object. If the latter also has a media.Video object
|
||||
# matching this video, then this function returns None and
|
||||
# nothing is deleted
|
||||
main_path = self.check_video_in_actual_dir(
|
||||
media_data_obj,
|
||||
video_obj,
|
||||
main_path,
|
||||
)
|
||||
|
||||
if main_path is not None \
|
||||
and os.path.isfile(main_path):
|
||||
|
||||
# Delete the description file
|
||||
os.remove(main_path)
|
||||
self.descrip_deleted_count += 1
|
||||
|
||||
# (Repeat for a file that might be in the sub-directory
|
||||
# '.data')
|
||||
subdir_path = video_obj.get_actual_path_in_subdirectory_by_ext(
|
||||
self.app_obj,
|
||||
'.description',
|
||||
)
|
||||
|
||||
subdir_path = self.check_video_in_actual_dir(
|
||||
subdir_path,
|
||||
video_obj,
|
||||
subdir_path,
|
||||
)
|
||||
|
||||
if subdir_path is not None \
|
||||
and os.path.isfile(subdir_path):
|
||||
|
||||
os.remove(subdir_path)
|
||||
self.descrip_deleted_count += 1
|
||||
|
||||
|
||||
def delete_json(self, media_data_obj):
|
||||
|
||||
"""Called by self.tidy_directory().
|
||||
|
||||
Checks all child videos of the specified media data object. If the
|
||||
associated metadata (JSON) file exists, delete it.
|
||||
|
||||
Args:
|
||||
|
||||
media_data_obj (media.Channel, media.Playlist or media.Folder):
|
||||
The media data object whose directory must be tidied up
|
||||
|
||||
"""
|
||||
|
||||
for video_obj in media_data_obj.compile_all_videos( [] ):
|
||||
|
||||
if video_obj.file_name is not None:
|
||||
|
||||
main_path = video_obj.get_actual_path_by_ext(
|
||||
self.app_obj,
|
||||
'.info.json',
|
||||
)
|
||||
|
||||
# If the video's parent container has an alternative download
|
||||
# destination set, we must check the corresponding media
|
||||
# data object. If the latter also has a media.Video object
|
||||
# matching this video, then this function returns None and
|
||||
# nothing is deleted
|
||||
main_path = self.check_video_in_actual_dir(
|
||||
media_data_obj,
|
||||
video_obj,
|
||||
main_path,
|
||||
)
|
||||
|
||||
if main_path is not None \
|
||||
and os.path.isfile(main_path):
|
||||
|
||||
# Delete the metadata file
|
||||
os.remove(main_path)
|
||||
self.json_deleted_count += 1
|
||||
|
||||
# (Repeat for a file that might be in the sub-directory
|
||||
# '.data')
|
||||
subdir_path = video_obj.get_actual_path_in_subdirectory_by_ext(
|
||||
self.app_obj,
|
||||
'.info.json',
|
||||
)
|
||||
|
||||
subdir_path = self.check_video_in_actual_dir(
|
||||
media_data_obj,
|
||||
video_obj,
|
||||
subdir_path,
|
||||
)
|
||||
|
||||
if subdir_path is not None \
|
||||
and os.path.isfile(subdir_path):
|
||||
|
||||
os.remove(subdir_path)
|
||||
self.json_deleted_count += 1
|
||||
|
||||
|
||||
def delete_xml(self, media_data_obj):
|
||||
|
||||
"""Called by self.tidy_directory().
|
||||
|
||||
Checks all child videos of the specified media data object. If the
|
||||
associated annotation file exists, delete it.
|
||||
|
||||
Args:
|
||||
|
||||
media_data_obj (media.Channel, media.Playlist or media.Folder):
|
||||
The media data object whose directory must be tidied up
|
||||
|
||||
"""
|
||||
|
||||
for video_obj in media_data_obj.compile_all_videos( [] ):
|
||||
|
||||
if video_obj.file_name is not None:
|
||||
|
||||
main_path = video_obj.get_actual_path_by_ext(
|
||||
self.app_obj,
|
||||
'.annotations.xml',
|
||||
)
|
||||
|
||||
# If the video's parent container has an alternative download
|
||||
# destination set, we must check the corresponding media
|
||||
# data object. If the latter also has a media.Video object
|
||||
# matching this video, then this function returns None and
|
||||
# nothing is deleted
|
||||
main_path = self.check_video_in_actual_dir(
|
||||
media_data_obj,
|
||||
video_obj,
|
||||
main_path,
|
||||
)
|
||||
|
||||
if main_path is not None \
|
||||
and os.path.isfile(main_path):
|
||||
|
||||
# Delete the annotation file
|
||||
os.remove(main_path)
|
||||
self.xml_deleted_count += 1
|
||||
|
||||
# (Repeat for a file that might be in the sub-directory
|
||||
# '.data')
|
||||
subdir_path = video_obj.get_actual_path_in_subdirectory_by_ext(
|
||||
self.app_obj,
|
||||
'.annotations.xml',
|
||||
)
|
||||
|
||||
subdir_path = self.check_video_in_actual_dir(
|
||||
media_data_obj,
|
||||
video_obj,
|
||||
subdir_path,
|
||||
)
|
||||
|
||||
if subdir_path is not None \
|
||||
and os.path.isfile(subdir_path):
|
||||
|
||||
os.remove(subdir_path)
|
||||
self.xml_deleted_count += 1
|
||||
|
||||
|
||||
def call_moviepy(self, video_obj, video_path):
|
||||
@ -1059,9 +1262,6 @@ class TidyManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 1063 call_moviepy')
|
||||
|
||||
try:
|
||||
clip = moviepy.editor.VideoFileClip(video_path)
|
||||
|
||||
@ -1075,7 +1275,7 @@ class TidyManager(threading.Thread):
|
||||
)
|
||||
|
||||
|
||||
def check_video_in_actual_dir(self, container_obj, video_obj, file_path):
|
||||
def check_video_in_actual_dir(self, container_obj, video_obj, delete_path):
|
||||
|
||||
"""Called by self.delete_video(), .delete_descrip(), .delete_json(),
|
||||
.delete_xml() and .delete_thumb().
|
||||
@ -1083,8 +1283,8 @@ class TidyManager(threading.Thread):
|
||||
If the video's parent container has an alternative download destination
|
||||
set, we must check the corresponding media data object. If the latter
|
||||
also has a media.Video object matching this video, then this function
|
||||
returns None and nothing is deleted. Otherwise, the specified file_path
|
||||
is returned, so it can be deleted.
|
||||
returns None and nothing is deleted. Otherwise, the specified
|
||||
delete_path is returned, so it can be deleted.
|
||||
|
||||
Args:
|
||||
|
||||
@ -1094,23 +1294,20 @@ class TidyManager(threading.Thread):
|
||||
video_obj (media.Video): A video contained in that channel,
|
||||
playlist or folder
|
||||
|
||||
file_path (str): The path to a file which the calling function
|
||||
delete_path (str): The path to a file which the calling function
|
||||
wants to delete
|
||||
|
||||
Returns:
|
||||
|
||||
The specified file_path if it can be deleted, or None if it should
|
||||
not be deleted
|
||||
The specified delete_path if it can be deleted, or None if it
|
||||
should not be deleted
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 1108 check_video_in_actual_dir')
|
||||
|
||||
if container_obj.dbid == container_obj.master_dbid:
|
||||
|
||||
# No alternative download destination to check
|
||||
return file_path
|
||||
return delete_path
|
||||
|
||||
else:
|
||||
|
||||
@ -1129,7 +1326,7 @@ class TidyManager(threading.Thread):
|
||||
|
||||
# There are no videos with the same name, so the file can be
|
||||
# deleted
|
||||
return file_path
|
||||
return delete_path
|
||||
|
||||
|
||||
def stop_tidy_operation(self):
|
||||
@ -1140,7 +1337,4 @@ class TidyManager(threading.Thread):
|
||||
Stops the tidy operation.
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('top 1144 stop_tidy_operation')
|
||||
|
||||
self.running_flag = False
|
||||
|
||||
@ -44,10 +44,6 @@ import utils
|
||||
from mainapp import _
|
||||
|
||||
|
||||
# Debugging flag (calls utils.debug_time at the start of every function)
|
||||
DEBUG_FUNC_FLAG = False
|
||||
|
||||
|
||||
# Classes
|
||||
|
||||
|
||||
@ -79,9 +75,6 @@ class UpdateManager(threading.Thread):
|
||||
|
||||
def __init__(self, app_obj, update_type):
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('uop 83 __init__')
|
||||
|
||||
super(UpdateManager, self).__init__()
|
||||
|
||||
# IV list - class objects
|
||||
@ -139,9 +132,6 @@ class UpdateManager(threading.Thread):
|
||||
Initiates the download.
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('uop 143 run')
|
||||
|
||||
if self.update_type == 'ffmpeg':
|
||||
self.install_ffmpeg()
|
||||
else:
|
||||
@ -163,9 +153,6 @@ class UpdateManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('uop 167 create_child_process')
|
||||
|
||||
info = preexec = None
|
||||
|
||||
if os.name == 'nt':
|
||||
@ -206,9 +193,6 @@ class UpdateManager(threading.Thread):
|
||||
application with the result of the update (success or failure).
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('uop 210 install_ffmpeg')
|
||||
|
||||
# Show information about the update operation in the Output Tab
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
@ -326,13 +310,11 @@ class UpdateManager(threading.Thread):
|
||||
application with the result of the update (success or failure).
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('uop 330 install_ytdl')
|
||||
|
||||
# Show information about the update operation in the Output Tab
|
||||
downloader = self.app_obj.get_downloader()
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
_('Starting update operation, installing/updating youtube-dl'),
|
||||
_('Starting update operation, installing/updating ' + downloader),
|
||||
)
|
||||
|
||||
# Prepare the system command
|
||||
@ -344,18 +326,25 @@ class UpdateManager(threading.Thread):
|
||||
cmd_list \
|
||||
= self.app_obj.ytdl_update_dict[self.app_obj.ytdl_update_current]
|
||||
|
||||
# Convert a path beginning with ~ (not on MS Windows)
|
||||
if os.name != 'nt':
|
||||
cmd_list[0] = re.sub('^\~', os.path.expanduser('~'), cmd_list[0])
|
||||
mod_list = []
|
||||
for arg in cmd_list:
|
||||
|
||||
# Substitute in the fork, if one is specified
|
||||
arg = self.app_obj.check_downloader(arg)
|
||||
# Convert a path beginning with ~ (not on MS Windows)
|
||||
if os.name != 'nt':
|
||||
arg = re.sub('^\~', os.path.expanduser('~'), arg)
|
||||
|
||||
mod_list.append(arg)
|
||||
|
||||
# Create a new child process using that command
|
||||
self.create_child_process(cmd_list)
|
||||
self.create_child_process(mod_list)
|
||||
|
||||
# Show the system command in the Output Tab
|
||||
space = ' '
|
||||
self.app_obj.main_win_obj.output_tab_write_system_cmd(
|
||||
1,
|
||||
space.join(cmd_list),
|
||||
space.join(mod_list),
|
||||
)
|
||||
|
||||
# So that we can read from the child process STDOUT and STDERR, attach
|
||||
@ -390,9 +379,14 @@ class UpdateManager(threading.Thread):
|
||||
# "The script youtube-dl is installed in '...' which is not
|
||||
# on PATH. Consider adding this directory to PATH..."
|
||||
if re.search('It looks like you installed', stdout) \
|
||||
or re.search('The script youtube-dl is installed', stdout):
|
||||
or re.search(
|
||||
'The script ' + downloader + ' is installed',
|
||||
stdout,
|
||||
):
|
||||
self.stderr_list.append(stdout)
|
||||
|
||||
else:
|
||||
|
||||
# Try to intercept the new version number for
|
||||
# youtube-dl
|
||||
self.intercept_version_from_stdout(stdout)
|
||||
@ -439,7 +433,7 @@ class UpdateManager(threading.Thread):
|
||||
# situations)
|
||||
if self.child_process is None:
|
||||
|
||||
msg = _('youtube-dl update did not start')
|
||||
msg = _('Update did not start')
|
||||
self.stderr_list.append(msg)
|
||||
self.app_obj.main_win_obj.output_tab_write_stdout(
|
||||
1,
|
||||
@ -488,9 +482,6 @@ class UpdateManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('uop 489 intercept_version_from_stdout')
|
||||
|
||||
substring = re.search(
|
||||
'Requirement already up\-to\-date.*\(([\d\.]+)\)\s*$',
|
||||
stdout,
|
||||
@ -525,9 +516,6 @@ class UpdateManager(threading.Thread):
|
||||
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('uop 526 is_child_process_alive')
|
||||
|
||||
if self.child_process is None:
|
||||
return False
|
||||
|
||||
@ -544,9 +532,6 @@ class UpdateManager(threading.Thread):
|
||||
Terminates the child process.
|
||||
"""
|
||||
|
||||
if DEBUG_FUNC_FLAG:
|
||||
utils.debug_time('uop 545 stop_update_operation')
|
||||
|
||||
if self.is_child_process_alive():
|
||||
|
||||
if os.name == 'nt':
|
||||
|
||||
315
tartube/utils.py
315
tartube/utils.py
@ -445,7 +445,7 @@ def convert_youtube_id_to_rss(media_type, youtube_id):
|
||||
|
||||
youtube_id (str): The YouTube channel or playlist ID
|
||||
|
||||
Return values:
|
||||
Returns:
|
||||
|
||||
The full URL for the RSS feed
|
||||
|
||||
@ -485,12 +485,12 @@ def convert_youtube_to_hooktube(url):
|
||||
return url
|
||||
|
||||
|
||||
def convert_youtube_to_invidious(url):
|
||||
def convert_youtube_to_invidious(app_obj, url):
|
||||
|
||||
"""Can be called by anything.
|
||||
|
||||
Converts a YouTube weblink to an Invidious weblink (but doesn't modify
|
||||
links to other sites.
|
||||
links to other sites).
|
||||
|
||||
Args:
|
||||
|
||||
@ -502,11 +502,12 @@ def convert_youtube_to_invidious(url):
|
||||
|
||||
"""
|
||||
|
||||
if re.search(r'^https?:\/\/(www)+\.youtube\.com', url):
|
||||
if re.search(r'^https?:\/\/(www)+\.youtube\.com', url) \
|
||||
and re.search('\w+\.\w+', app_obj.custom_invidious_mirror):
|
||||
|
||||
url = re.sub(
|
||||
r'youtube\.com',
|
||||
'invidio.us',
|
||||
app_obj.custom_invidious_mirror,
|
||||
url,
|
||||
# Substitute first occurence only
|
||||
1,
|
||||
@ -775,31 +776,87 @@ def find_thumbnail(app_obj, video_obj, temp_dir_flag=False):
|
||||
for ext in formats.IMAGE_FORMAT_LIST:
|
||||
|
||||
# Look in Tartube's permanent data directory
|
||||
path = video_obj.get_actual_path_by_ext(app_obj, ext)
|
||||
|
||||
if os.path.isfile(path):
|
||||
return path
|
||||
normal_path = video_obj.check_actual_path_by_ext(app_obj, ext)
|
||||
if normal_path is not None:
|
||||
return normal_path
|
||||
|
||||
elif temp_dir_flag:
|
||||
|
||||
# Look in temporary data directory
|
||||
data_dir_len = len(app_obj.downloads_dir)
|
||||
|
||||
temp_path = app_obj.temp_dl_dir + path[data_dir_len:]
|
||||
temp_path = video_obj.get_actual_path_by_ext(app_obj, ext)
|
||||
temp_path = app_obj.temp_dl_dir + temp_path[data_dir_len:]
|
||||
if os.path.isfile(temp_path):
|
||||
return temp_path
|
||||
|
||||
# Catch YouTube .jpg thumbnails, in the form .jpg?...
|
||||
normal_path = video_obj.get_actual_path_by_ext(app_obj, '.jpg*')
|
||||
for glob_path in glob.glob(normal_path):
|
||||
if os.path.isfile(glob_path):
|
||||
return glob_path
|
||||
|
||||
if temp_dir_flag:
|
||||
|
||||
temp_path = video_obj.get_actual_path_by_ext(app_obj, '.jpg*')
|
||||
temp_path = app_obj.temp_dl_dir + temp_path[data_dir_len:]
|
||||
|
||||
for glob_path in glob.glob(temp_path):
|
||||
if os.path.isfile(glob_path):
|
||||
return glob_path
|
||||
|
||||
|
||||
# No matching thumbnail found
|
||||
return None
|
||||
|
||||
|
||||
def find_thumbnail_restricted(app_obj, video_obj):
|
||||
|
||||
"""Called by mainapp.TartubeApp.update_video_when_file_found().
|
||||
|
||||
Modified version of utils.find_thumbnail().
|
||||
|
||||
Returns the path of the thumbnail in the same directory as its video. The
|
||||
path is returned as a list, so the calling code can convert it into the
|
||||
equivalent path in the '.thumbs' subdirectory.
|
||||
|
||||
Args:
|
||||
|
||||
app_obj (mainapp.TartubeApp): The main application
|
||||
|
||||
video_obj (media.Video): The video object handling the downloaded video
|
||||
|
||||
Returns:
|
||||
|
||||
return_list (list): A list whose items, when combined, will be the full
|
||||
path to the thumbnail file. If no thumbnail file was found, an
|
||||
empty list is returned
|
||||
|
||||
"""
|
||||
|
||||
for ext in formats.IMAGE_FORMAT_LIST:
|
||||
|
||||
actual_dir = video_obj.parent_obj.get_actual_dir(app_obj)
|
||||
test_path = os.path.abspath(
|
||||
os.path.join(
|
||||
actual_dir,
|
||||
video_obj.file_name + ext,
|
||||
),
|
||||
)
|
||||
|
||||
if os.path.isfile(test_path):
|
||||
return [ actual_dir, video_obj.file_name + ext ]
|
||||
|
||||
# No matching thumbnail found
|
||||
return []
|
||||
|
||||
|
||||
def find_thumbnail_webp(app_obj, video_obj):
|
||||
|
||||
"""Called by tidy.TidyManager.delete_webp().
|
||||
"""Can be called by anything.
|
||||
|
||||
In June 2020, YouTube started serving .webp thumbnails. At the time of
|
||||
writing (v2.1.027), Gtk can't display them. A youtube-dl fix is expected,
|
||||
which will convert .webp thumbnails to .jpg; in anticipation of that, we
|
||||
add an option to remove .webp files.
|
||||
In June 2020, YouTube started serving .webp thumbnails. Gtk cannot display
|
||||
them, so Tartube typically converts themto .jpg.
|
||||
|
||||
This is a modified version of utils.find_thumbnail(), which looks for
|
||||
thumbnails in the .webp or malformed .jpg format, and return the path to
|
||||
@ -817,16 +874,36 @@ def find_thumbnail_webp(app_obj, video_obj):
|
||||
|
||||
"""
|
||||
|
||||
path = video_obj.get_actual_path_by_ext(app_obj, '.webp')
|
||||
if os.path.isfile(path):
|
||||
return path
|
||||
for ext in ('.webp', '.jpg'):
|
||||
|
||||
# malformed .jpg thumbnail files have an extension .jpg?sqp=-XXX, where XXX
|
||||
# is a large number of random characters
|
||||
path = video_obj.get_actual_path_by_ext(app_obj, '.jpg?*')
|
||||
for actual_path in glob.glob(path):
|
||||
if os.path.isfile(actual_path):
|
||||
return actual_path
|
||||
main_path = video_obj.get_actual_path_by_ext(app_obj, ext)
|
||||
if os.path.isfile(main_path) \
|
||||
and app_obj.ffmpeg_manager_obj.is_webp(main_path):
|
||||
return main_path
|
||||
|
||||
# The extension may be followed by additional characters, e.g.
|
||||
# .jpg?sqp=-XXX (as well as several other patterns)
|
||||
for actual_path in glob.glob(main_path + '*'):
|
||||
if os.path.isfile(actual_path) \
|
||||
and app_obj.ffmpeg_manager_obj.is_webp(actual_path):
|
||||
return actual_path
|
||||
|
||||
subdir_path = video_obj.get_actual_path_in_subdirectory_by_ext(
|
||||
app_obj,
|
||||
ext,
|
||||
)
|
||||
|
||||
if os.path.isfile(subdir_path) \
|
||||
and app_obj.ffmpeg_manager_obj.is_webp(subdir_path):
|
||||
return subdir_path
|
||||
|
||||
for actual_path in glob.glob(subdir_path + '*'):
|
||||
if os.path.isfile(actual_path) \
|
||||
and app_obj.ffmpeg_manager_obj.is_webp(actual_path):
|
||||
return actual_path
|
||||
|
||||
# No webp thumbnail found
|
||||
return None
|
||||
|
||||
|
||||
def format_bytes(num_bytes):
|
||||
@ -912,11 +989,8 @@ divert_mode=None):
|
||||
# If actually downloading videos, use (or create) an archive file so that,
|
||||
# if the user deletes the videos, youtube-dl won't try to download them
|
||||
# again
|
||||
# We don't use an archive file when:
|
||||
# 1. Downloading into a system folder
|
||||
# 2. Checking for missing videos
|
||||
if not missing_video_check_flag \
|
||||
and (
|
||||
# We don't use an archive file when downloading into a system folder
|
||||
if (
|
||||
not dl_classic_flag and app_obj.allow_ytdl_archive_flag \
|
||||
or dl_classic_flag and app_obj.classic_ytdl_archive_flag
|
||||
):
|
||||
@ -958,9 +1032,17 @@ divert_mode=None):
|
||||
|
||||
# Supply youtube-dl with the path to the ffmpeg/avconv binary, if the
|
||||
# user has provided one
|
||||
if app_obj.ffmpeg_path is not None:
|
||||
# If both paths have been set, prefer ffmpeg, unless the 'prefer_avconv'
|
||||
# download option had been specified
|
||||
if '--prefer-avconv' in options_list and app_obj.avconv_path is not None:
|
||||
options_list.append('--ffmpeg-location')
|
||||
options_list.append(app_obj.avconv_path)
|
||||
elif app_obj.ffmpeg_path is not None:
|
||||
options_list.append('--ffmpeg-location')
|
||||
options_list.append(app_obj.ffmpeg_path)
|
||||
elif app_obj.avconv_path is not None:
|
||||
options_list.append('--ffmpeg-location')
|
||||
options_list.append(app_obj.avconv_path)
|
||||
|
||||
# Convert a YouTube URL to an alternative YouTube front-end, if required
|
||||
source = media_data_obj.source
|
||||
@ -968,14 +1050,14 @@ divert_mode=None):
|
||||
if divert_mode == 'hooktube':
|
||||
source = convert_youtube_to_hooktube(source)
|
||||
elif divert_mode == 'invidious':
|
||||
source = convert_youtube_to_invidious(source)
|
||||
source = convert_youtube_to_invidious(app_obj, source)
|
||||
elif divert_mode == 'custom' \
|
||||
and app_obj.custom_dl_divert_website is not None \
|
||||
and len(app_obj.custom_dl_divert_website) > 2:
|
||||
source = convert_youtube_to_other(app_obj, source)
|
||||
|
||||
# Convert a path beginning with ~ (not on MS Windows)
|
||||
ytdl_path = app_obj.ytdl_path
|
||||
ytdl_path = app_obj.check_downloader(app_obj.ytdl_path)
|
||||
if os.name != 'nt':
|
||||
ytdl_path = re.sub('^\~', os.path.expanduser('~'), ytdl_path)
|
||||
|
||||
@ -1062,6 +1144,100 @@ def is_youtube(url):
|
||||
return False
|
||||
|
||||
|
||||
def move_metadata_to_subdir(app_obj, video_obj, ext):
|
||||
|
||||
"""Can be called by anything.
|
||||
|
||||
Moves a description, JSON or annotations file from the same directory as
|
||||
its video, into the subdirectory '.data'.
|
||||
|
||||
Args:
|
||||
|
||||
app_obj (mainapp.TartubeApp): The main application
|
||||
|
||||
video_obj (media.Video): The file's parent video
|
||||
|
||||
ext (str): The file extension, which will be one of '.description',
|
||||
'.info.json' or '.annotations.xml'
|
||||
|
||||
"""
|
||||
|
||||
main_path = video_obj.get_actual_path_by_ext(app_obj, '.description')
|
||||
subdir = os.path.abspath(
|
||||
os.path.join(
|
||||
video_obj.parent_obj.get_actual_dir(app_obj),
|
||||
app_obj.metadata_sub_dir,
|
||||
),
|
||||
)
|
||||
|
||||
subdir_path = video_obj.get_actual_path_in_subdirectory_by_ext(
|
||||
app_obj,
|
||||
'.description',
|
||||
)
|
||||
|
||||
if os.path.isfile(main_path) and not os.path.isfile(subdir_path):
|
||||
|
||||
try:
|
||||
if not os.path.isdir(subdir):
|
||||
os.makedirs(subdir)
|
||||
|
||||
# (os.rename sometimes fails on external hard drives; this
|
||||
# is safer)
|
||||
shutil.move(main_path, subdir_path)
|
||||
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
def move_thumbnail_to_subdir(app_obj, video_obj):
|
||||
|
||||
"""Can be called by anything.
|
||||
|
||||
Moves a thumbnail file from the same directory as its video, into the
|
||||
subdirectory '.thumbs'.
|
||||
|
||||
Args:
|
||||
|
||||
app_obj (mainapp.TartubeApp): The main application
|
||||
|
||||
video_obj (media.Video): The file's parent video
|
||||
|
||||
"""
|
||||
|
||||
path_list = utils.find_thumbnail_restricted(app_obj, video_obj)
|
||||
if path_list:
|
||||
|
||||
main_path = os.path.abspath(
|
||||
os.path.join(
|
||||
path_list[0], path_list[1],
|
||||
),
|
||||
)
|
||||
|
||||
subdir = os.path.abspath(
|
||||
os.path.join(
|
||||
path_list[0], app_obj.thumbs_sub_dir,
|
||||
),
|
||||
)
|
||||
|
||||
subdir_path = os.path.abspath(
|
||||
os.path.join(
|
||||
path_list[0], app_obj.thumbs_sub_dir, path_list[1],
|
||||
),
|
||||
)
|
||||
|
||||
if os.path.isfile(main_path) \
|
||||
and not os.path.isfile(subdir_path):
|
||||
|
||||
try:
|
||||
if not os.path.isdir(subdir):
|
||||
os.makedirs(subdir)
|
||||
|
||||
shutil.move(main_path, subdir_path)
|
||||
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
def open_file(uri):
|
||||
|
||||
"""Can be called by anything.
|
||||
@ -1086,9 +1262,15 @@ def parse_ytdl_options(options_string):
|
||||
|
||||
"""Called by options.OptionsParser.parse() or info.InfoManager.run().
|
||||
|
||||
Also called by process.ProcessManager.__init__, to parse FFmpeg command-
|
||||
line options on the same basis.
|
||||
|
||||
Parses the 'extra_cmd_string' option, which can contain arguments inside
|
||||
double quotes "..." (arguments that can therefore contain whitespace)
|
||||
|
||||
If options_string contains newline characters, then it terminates an
|
||||
argument, closing newline character or not.
|
||||
|
||||
Args:
|
||||
|
||||
options_string (str): A string containing various youtube-dl
|
||||
@ -1100,31 +1282,33 @@ def parse_ytdl_options(options_string):
|
||||
|
||||
"""
|
||||
|
||||
# Set a flag for an item beginning with double quotes, and reset it for an
|
||||
# item ending in double quotes
|
||||
quote_flag = False
|
||||
# Temporary list to hold such quoted arguments
|
||||
quote_list = []
|
||||
# Add options, one at a time, to a list
|
||||
return_list = []
|
||||
|
||||
return_string = ''
|
||||
for item in options_string.split():
|
||||
for line in options_string.splitlines():
|
||||
|
||||
quote_flag = (quote_flag or item[0] == "\"")
|
||||
# Set a flag for an item beginning with double quotes, and reset it for
|
||||
# an item ending in double quotes
|
||||
quote_flag = False
|
||||
# Temporary list to hold such quoted arguments
|
||||
quote_list = []
|
||||
|
||||
if quote_flag:
|
||||
quote_list.append(item)
|
||||
else:
|
||||
return_list.append(item)
|
||||
for item in line.split():
|
||||
|
||||
if quote_flag and item[-1] == "\"":
|
||||
quote_flag = (quote_flag or item[0] == "\"")
|
||||
|
||||
# Special case mode is over
|
||||
return_list.append(" ".join(quote_list)[1:-1])
|
||||
if quote_flag:
|
||||
quote_list.append(item)
|
||||
else:
|
||||
return_list.append(item)
|
||||
|
||||
quote_flag = False
|
||||
quote_list = []
|
||||
if quote_flag and item[-1] == "\"":
|
||||
|
||||
# Special case mode is over
|
||||
return_list.append(" ".join(quote_list)[1:-1])
|
||||
|
||||
quote_flag = False
|
||||
quote_list = []
|
||||
|
||||
return return_list
|
||||
|
||||
@ -1177,6 +1361,39 @@ def strip_whitespace(string):
|
||||
return string
|
||||
|
||||
|
||||
def strip_whitespace_multiline(string):
|
||||
|
||||
"""Can be called by anything.
|
||||
|
||||
An extended version of utils.strip_whitepspace.
|
||||
|
||||
Divides a string into lines, removes empty lines, removes any leading/
|
||||
trailing whitespace from each line, then combines the lines back into a
|
||||
single string (with lines separated by newline characters).
|
||||
|
||||
Args:
|
||||
|
||||
string (str): The string to convert
|
||||
|
||||
Returns:
|
||||
|
||||
The converted string
|
||||
|
||||
"""
|
||||
|
||||
line_list = string.splitlines()
|
||||
mod_list = []
|
||||
|
||||
for line in line_list:
|
||||
line = re.sub(r'^\s+', '', line)
|
||||
line = re.sub(r'\s+$', '', line)
|
||||
|
||||
if re.search('\S', line):
|
||||
mod_list.append(line)
|
||||
|
||||
return "\n".join(mod_list)
|
||||
|
||||
|
||||
def tidy_up_container_name(string, max_length):
|
||||
|
||||
"""Called by mainapp.TartubeApp.on_menu_add_channel(),
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user