266 Commits (master)

Author SHA1 Message Date
Felix Stupp c97559ed62
Reworked ExtractedData classes, split into Offline & Online 2 years ago
Felix Stupp 487b126fe1
Add aggregated extraction support 2 years ago
Felix Stupp 0d75fa72bd
Add TMDB extraction support 2 years ago
Felix Stupp 1139219e23
Add support for TVMaze extraction 2 years ago
Felix Stupp 9ffb34b972
Use ExtractedDataLight where data cannot be supplied 2 years ago
Felix Stupp d6768135f3
MediaExtractor: Fix type hints to Generic from super class 2 years ago
Felix Stupp 9c7fd0d4cd
Extract media_extract_uri_new from media_extract_uri 2 years ago
Felix Stupp 0554dcff18
Extract collection_extract_uri_new from collection_extract_uri 2 years ago
Felix Stupp af1c279eb3
YouTubeCollectionExtractor: Set creator if already extracted 2 years ago
Felix Stupp 9b31fb73cb
YouTubeCollectionExtractor: Set a release_date determined by first video 2 years ago
Felix Stupp cacc6e9a05
YouTubeCollectionExtractor: Reformat code 2 years ago
Felix Stupp 4a690688f6
YouTubeCollectionExtractor: Use sorting_method to sort channel videos by date 2 years ago
Felix Stupp 8f100ad917
YouTubeCollectionExtractor: Extract description 2 years ago
Felix Stupp 7913cd3561
YouTubeCollectionExtractor: allow partial-only updates on channels 2 years ago
Felix Stupp 2dd0b575fa
TT-RSS: Set collection author & watch in order auto val 2 years ago
Felix Stupp e4a50288d6
Enable mark_as_read for TT-RSS extractor 2 years ago
Felix Stupp 081964a336
Add many properties to MediaCollection 2 years ago
Felix Stupp fc35a62871
Extract set_watch_in_order_auto to MediaCollection 2 years ago
Felix Stupp 97796ce499
Remove unused methods from PreferenceScore
- order_by_score & get_first_by_score
- easily replaceable by sorted / min calls
2 years ago
Felix Stupp 8cedaee7ba
generate_preference_score: Add static scores
for following properties:
- id (for consistent order)
- age
- began to watch
- count of tags
- times pinned
2 years ago
Felix Stupp 3c64bc6f66
generate_preference_score: Add auto tag for extractor 2 years ago
Felix Stupp 0e266c55b6
generate_preference_list: Use set instead of recalling object_gen
- removes advanced feature of listing blocked videos ahead
  - was not practical for UI as videos were displayed which "require" others
- using a set to manage still to consider videos makes method way faster
2 years ago
Felix Stupp bd77ed7fb0
generate_preference_list: Move var res_ids to appropiate section 2 years ago
Felix Stupp 0599b03daa
generate_preference_list: Separate sections of god method 2 years ago
Felix Stupp 1145a09e3e
generate_preference_list: Add "collection tags" by iter collections
itering collections than elements for their collections is most of the
time faster as mostly all collections are involved
2 years ago
Felix Stupp 748201ec2b
generate_preference_list: Revert Tag table auto incr value
- not reverted by default when ROLLBACK on MariaDB is issued
- resetting to 1 works because DB will still ensure no duplicates occure
2 years ago
Felix Stupp 7b96a6e2c6
Add MediaCollectionLink.desc_sort_key 2 years ago
Felix Stupp 0b44e77677
Reorder attributes of MediaCollection 2 years ago
Felix Stupp 7248ef96cd
entertainment_decider/models: Add return types for mypy 2 years ago
Felix Stupp e53ffccef3
Reorder & categorize MediaElement attributes 2 years ago
Felix Stupp 9ea4a728d6
Rework Tagable implementation
Split into orm compatible & non-compatible methods

FIX Tagable on MediaElement

FIX Tagable MediaColl
2 years ago
Felix Stupp c7b63dfedb
models: Fix build of custom table defs
- missing .items() call for "for" with dict
- missing db session on setup
2 years ago
Felix Stupp bbaa79f20a
models:table_exists: use pony-given approach 2 years ago
Felix Stupp 51a13abb88
entertainment_decider/models: Remove TODO about static table names 2 years ago
Felix Stupp fd7d413fe2
Add MediaElement properties detected_{creators,playlists} 2 years ago
Felix Stupp c8d755770f
Allow PreferenceScore.adapt_score to apply only flat 2 years ago
Felix Stupp c766ca1c58
Reformat PreferenceScore.adapt_score 2 years ago
Felix Stupp ee9f89a87d
Tagable: Implement tag_hierachy & all_tags separately
Other implementation of all_tags makes it much, much faster
2 years ago
Felix Stupp d56cf1ae94
TagTreeElement.shareScore: Filter tags for score sharing by use_for_preferences 2 years ago
Felix Stupp 760010e743
Rewrite PreferenceScore.(from|to)_base64 to method chaining
Solves mypy issues while looking clean
2 years ago
Felix Stupp a039aafff2
PreferenceScore.calculate_score improved 2 years ago
Felix Stupp cf80adda94
Declare MediaCollection.release_date nullable 2 years ago
Felix Stupp 51c1ee5c08
Add missing Optional on _load_object & use explcit is None checks 2 years ago
Felix Stupp 2214df4561
Add capability to inject episodes from collection updates 2 years ago
Felix Stupp 7c96aaa6a7
Avoid circular dependency by moving import into function 2 years ago
Felix Stupp 39b3ac278b
extractors: Add & Fix return types
tt_rss's _update_object_raw should have never returned a str
2 years ago
Felix Stupp 7c5d8e7748
entertainment_decider/models: Clean code for mypy
- add return types
- rename used TypeVar to avoid override
- adapt type of MediaCollection.creator to reflect Optionality
2 years ago
Felix Stupp c59000847d
Decrease external cache time for ytdl to 1 day 2 years ago
Felix Stupp 94fc20cb89
Declare entertainment_decider/extractors a module
by creating an empty __init__.py
2 years ago
Felix Stupp 6ab8781520
Add way to inject object while extracting
To save extracting work, if e.g. updating a collection also receives
data about medias in it (which then do not need any more requests)
2 years ago
Felix Stupp ae047a0c47
extractors/generic: Clean code
- Reformat
- Add return types
- Remove unrequired imports
2 years ago
Felix Stupp 5d68e03971
common: Add helper for string trimming
like defined in PEP 257
2 years ago
Felix Stupp 43e1d6902f
common: Add helper to convert date to datetime 2 years ago
Felix Stupp ddb248ca62
common: Add type hints to call helper 2 years ago
Felix Stupp 051d4a7ccd
Added logging to suitable extractor search 2 years ago
Felix Stupp f8b9825a2a
TtRssCollectionExtractor: Increase cache timeout to 15 minutes 2 years ago
Felix Stupp e486dccf8e
Add capability to mark TT-RSS feed items as read 2 years ago
Felix Stupp 321847446b
Add sorting mechanism post collection update
e.g. sort some collections strictly on release date
(if collection author is not capable of sorting them ...)
2 years ago
Felix Stupp f51cbc31c3
CollectionExtractor._add_episode: Only log on link change 2 years ago
Felix Stupp a1b64d8644
extractors/collection/base: Reformat code 2 years ago
Felix Stupp ed8fc4439e
Introduce dynamic cache calculation for YouTube collection 2 years ago
Felix Stupp 49cff6f300
uri_suitable: Explicitly check for "is not None" & use helper methods
- helper for convertion from boolean to SuitableLevel
2 years ago
Felix Stupp 54748a9540
_check_cache_expired modified to be able to access full object
for more advanced calculations about caching
2 years ago
Felix Stupp 2e97563e0e
YtdlMediaExtractor._update_object_raw: Add return type 2 years ago
Felix Stupp bdfcd43a5f
models: Extract common Uri methods to own class 2 years ago
Felix Stupp f282ad7ff1
Add property MediaElement.average_release_per_week{,_now} 2 years ago
Felix Stupp 9e4b2bab5e
Add property MediaElement.started 2 years ago
Felix Stupp 3f68d4c5fd
MediaThumbnail.data: Add reason for lazy setting 2 years ago
Felix Stupp a9cae7ab1e
MediaCollection: Add indices to keep_updated, pinned, watch_in_order
for faster DB requests
2 years ago
Felix Stupp abf01da286
MediaElement: Add indeces to watched & ignored & compositum of both 2 years ago
Felix Stupp 5267f98cad
Add MediaCollection.description attribute / column 2 years ago
Felix Stupp 214cfdffba
MediaCollection.add_episode: automatically mark new episodes or ignored colls as ignored 2 years ago
Felix Stupp fac70231e8
MediaCollection.add_episde: detect change & act accordingly
- only apply changes if episode is new or new data is given
- flush only if changes are given
- return link only on changes / creation
2 years ago
Felix Stupp 4a0a6b918a
models.py: Reformat code 2 years ago
Felix Stupp 52a5aa29d7
models: Make table names non-hardcoded 2 years ago
Felix Stupp 66c37e72aa
models: Extract table name "element_lookup_cache" to constant 2 years ago
Felix Stupp f810591c2c
Revert "Add ConsideredMediaGenerator"
This reverts commit a5844cc424,
because developed method was not fast enough & it was faster to use a
SQL native method developed afterwards
2 years ago
Felix Stupp a5844cc424
Add ConsideredMediaGenerator
Developed for faster method to find considerable media
2 years ago
Felix Stupp 7e5e58c487
models.py: Reformated code 2 years ago
Felix Stupp ef09aea0b5
MediaElement: Add index to release_date 2 years ago
Felix Stupp e5dc2db760
MediaElement.watched: Fix column name 2 years ago
Felix Stupp 7ea87a1e0d
Use element_lookup_cache
- caches by what elements other elements are blocked
- resolves lookups for ordered playlists
2 years ago
Felix Stupp 05fcd9fb6a
MediaElement: add direct blockings 2 years ago
Felix Stupp 53185505b1
MediaElement.left_length: return 0 if watched 2 years ago
Felix Stupp a923d99e82
MediaElement.can_considered: Only request data for ordered collections 2 years ago
Felix Stupp cc8cb6b072
models: Redefine Query to be more usable for type hinting 2 years ago
Felix Stupp 85363aed4b
MediaElement.can_considered: verify video already released 2 years ago
Felix Stupp 5c97fe4b77
Implemented element_lookup_cache
- not used for now
2 years ago
Felix Stupp 96774bc43e
Add support for custom tables 2 years ago
Felix Stupp 2a7654bf9f
Add helper annotation for profiling functions 2 years ago
Felix Stupp babdf8ff64
extr/media/youtube: Reraise exceptions from extraction as ExtractionErrors 2 years ago
Felix Stupp 33ac573a57
extr/media/youtube: Ignore live streams until they are available as VOD 2 years ago
Felix Stupp 922c9fd5a2
extr/media/youtube: Rename extractor to "youtube" 2 years ago
Felix Stupp 0debf4d875
extr/media/youtube: support embed & shorts video links 2 years ago
Felix Stupp 5479db90b8
extr/media/youtube: Also support youtube-nocookie links 2 years ago
Felix Stupp 35024597df
extr/media/youtube: Rewrite URI so extractor works deteministicly 2 years ago
Felix Stupp 6bb65981c9
extr/media/youtube: Explicit check for suitable URI 2 years ago
Felix Stupp 902dd1a6bf
models:MediaCollection: Add creator / created_collections relation 2 years ago
Felix Stupp 6f9adbfc25
models: Make Query Type hacky typeable 2 years ago
Felix Stupp 96e12f6fe8
extr/coll,media/youtube: Move setting URLs to media extractor 2 years ago
Felix Stupp 4e62a3fe70
extr/media/youtube: Extract description 2 years ago
Felix Stupp c9c60d4e2b
extr/media/ytdl: Extract description 2 years ago
Felix Stupp 2e9df67e86
models: Add optional description to MediaElement 2 years ago
Felix Stupp 683a7a2555
extr/media/ytdl: Add thumbnail of video 2 years ago
Felix Stupp cfa3012798
extr/media/ytdl: Restrict to https?:// urls 2 years ago
Felix Stupp f0af32c099
Extract media thumbnail automatically from YouTube 2 years ago
Felix Stupp dfee06106f
ent_dec/extr/med/base:MediaExtractor._get_auth_d: Fix not using TypeVar 2 years ago
Felix Stupp f236db4084
Format code
- server/entertainment_decider/common.py
- server/entertainment_decider/extractors/all/tt_rss.py
- server/entertainment_decider/extractors/all/ytdl.py
- server/entertainment_decider/extractors/collection/__init__.py
- server/entertainment_decider/extractors/collection/tt_rss.py
- server/entertainment_decider/extractors/collection/youtube.py
- server/entertainment_decider/extractors/generic.py
- server/entertainment_decider/extractors/helpers.py
- server/entertainment_decider/extractors/media/__init__.py
- server/entertainment_decider/extractors/media/base.py
- server/entertainment_decider/extractors/media/youtube.py
- server/entertainment_decider/extractors/media/ytdl.py

Applied automatically using black
2 years ago
Felix Stupp b7cbcbd540
Reformat models.py 2 years ago
Felix Stupp 44eb0850f0
Add thumbnail support to MediaElements 2 years ago
Felix Stupp bd72ad77bb
Reformat code (partially using black) 2 years ago
Felix Stupp d73cda2e5b
MediaCollection.completed: Rewrite to use orm/db mapping 3 years ago
Felix Stupp b42611f38c
MediaCollection.next_episode: Moved operation into orm/db 3 years ago
Felix Stupp 043c49c0e1
MediaCollection.__to_watch_epsiodes: Replaced watched/ignored check with skip_over 3 years ago
Felix Stupp 73a48f502c
Added PreferenceScore.get_first_by_score and used for performance 3 years ago
Felix Stupp d0c2aab529
Added youtube media extractor based on new version of youtubesearchpython 3 years ago
Felix Stupp 075b0a544c
extractors.SuitableLevel: Added methods for fancy tenary's 3 years ago
Felix Stupp 9d0fc6b08c
PreferenceScore: Add methods to pickle to/from json/base64 3 years ago
Felix Stupp 840a85c62f
MediaCollection.__to_watch_episodes: Add pony's Query as return type 3 years ago
Felix Stupp 0c205b2a43
MediaCollectionLink: Extract sort_key as replacement for natural_sort 3 years ago
Felix Stupp 20ce11b9e7
TagRootElement.share_score: Also abort if points to share are 0 3 years ago
Felix Stupp 5743b70d96
models ignored_recurive: Reworked for simplicity 3 years ago
Felix Stupp 099b3423e9
models ignored_recursive: Fix searching links for this episode 3 years ago
Felix Stupp 862cf78c74
models: Fix column name of MediaElement.ignored 3 years ago
Felix Stupp a51aa48941
extractors/collection: Extract _add_episode into base 3 years ago
Felix Stupp 629e125838
models: MediaCollection: Increase effiency of completed property 3 years ago
Felix Stupp 0c1b82a9cd
Add general overall dashboard 3 years ago
Felix Stupp 39d8c4f535
models: Add MediaCollection.pinned property 3 years ago
Felix Stupp 87b4fb03ea
models: Fix MediaElement.can_considered check for being next episode 3 years ago
Felix Stupp 5b78e8120c
add MediaCollection|MediaElement.add_single_uri and rewrite appropiate usages 3 years ago
Felix Stupp c77e0ac81e
entertainment_decider/extractors/generic: Add debug log for skipped elements due to cache timeout 3 years ago
Felix Stupp 18ec39e7b2
extractors/collection/tt_rss: Remove debug priting for labels 3 years ago
Felix Stupp b982f7b5bb
extractors/collection/tt_rss: Decrease cache timeout to 10 minutes 3 years ago
Felix Stupp c99c69b7df
extractors/collection/*: Fix check for cache timeout 3 years ago
Felix Stupp 6b45d0fbd1
extractors/{collection,media}: Add *_update method 3 years ago
Felix Stupp 4d5f5ea4d4
extractors/{collection,media}: Extract *_expect_extractor "partial" method 3 years ago
Felix Stupp bf14abc887
models: Fix MediaElement.can_considered to check if next episode link is None 3 years ago
Felix Stupp f989f75806
models: Rewrite MediaElement.ignored_recursive to use fast SQL request to count 3 years ago
Felix Stupp bf548f13ad
models: let MediaElement.left_length return 0 if video already watched 3 years ago
Felix Stupp 26ea1d6451
models: Add missing "super_tags" property to Tag 3 years ago
Felix Stupp d32e8feddd
models: Fix Tagable to correctly return TagTreeElement's and not Tags 3 years ago
Felix Stupp 37e7d602e8
models.py: Fix Tagable.assigned_tags to raise NotImplementedError 3 years ago
Felix Stupp 04d73f253d
models: Fix points on (TagRootElement|TagTreeElement).share_score to be float 3 years ago
Felix Stupp 6312143abb
Refractor/Extend tag system and add auto recommendation algorithm 3 years ago
Felix Stupp 6330443876
Allow unextracted elements and collections to exists, add/use reflecting property 3 years ago
Felix Stupp ad5a504263
extractors/collection/youtube: Normalize playlist titles as well 3 years ago
Felix Stupp 1bca3104bd
Normalize auto-generated collection titles 3 years ago
Felix Stupp d0b3795484
extractors/generic: Add Optional to load_* type hints 3 years ago
Felix Stupp b19bfbb8a4
extractors/collection: Also configure newly created collection object 3 years ago
Felix Stupp 4adb0484f3
extractors/collection: Extract method __configure_collection 3 years ago