Klient Hydrus Network to aplikacja napisana dla anonimowych użytkowników i innych entuzjastów mediów w Internecie, którzy mają duże kolekcje obrazów. Organizuje pliki w wewnętrznej bazie danych i przegląda je za pomocą tagów, zamiast folderów, trochę jak misz-masz na pulpicie. Tagi i pliki można udostępniać anonimowo za pośrednictwem niestandardowych serwerów, które może uruchamiać każdy użytkownik. Wszystko jest darmowe, prywatność jest najważniejsza, a kod źródłowy jest dołączony do wydania. Wersje są dostępne dla systemów Windows, Linux i macOS.
Obecnie możliwe do importu typy plików to:
obrazy - jpg, gif (w tym animowane), png (w tym animowane!), tiff, webp, bmp
fixed a stupid oversight with last week's "move page focus left/right after closing tab" thing where it was firing even when the page closed was not the current tab!! it now correctly only moves your focus if you close the current tab, not if you just middle click some other one
fixed the share->export files menu command not showing if you right-clicked on just one file
cleaned some of the broader thumbnail menu code, separating the 'stuff to show if we have a focus' and 'stuff to show if we have a selection'; the various 'manage' commands now generally show even if there is no current 'focus' in the preview (which happens if you select with ctrl+click or ctrl+a and then right-click in whitespace)
the 'migrate tags' dialog now allows you to filter the sibling or parent pairs by whether the child/worse or parent/ideal tag has actual mapping counts on an arbitrary tag service. some new unit tests ensure this capability
fixed an error in the duplicate metadata merge system where if files were exchanging known URLs, and one of those URLs was not actually an URL (e.g. it was garbage data, or human-entered 'location' info), a secondary system that tried to merge correlated domain-based timestamps was throwing an exception
to reduce comma-confusion, the template for 'show num files and import status' on page names is now "name - (num_files - import_status)"
the option that governs whether page names have the file count after them (under options->gui pages) has a new choice--'show for all pages, but only if greater than zero'--which is now the default for new users
some boring code cleanup:
broke up the over-coupled 'migrate tags' unit tests into separate content types and the new count-filtering stuff
cleaned up the 'share' menu construction code--it was messy after some recent rewrites
added some better error handling around some of the file/thumbnail path fetching/regen routines
client api:
the client api gets a new permissions state this week: the permissions structure you edit for an access key can now be (and, as a convenient default, starts as) a simple 'permits everything' state. if the permissions are set to 'permit everything', then this overrules all the specific rules and tag search filter gubbins. nice and simple, and a permissions set this way will automatically inherit new permissions in the future. any api access keys that have all the permissions up to 'edit ratings' will be auto-updated to 'permits everything' and you will get an update saying this happened--check your permissions in review services if you need finer control
added a new permission, 13, for 'see local paths'
added /get_files/file_path, which fetches the local path of a file. it needs the new permission
added /get_files/thumbnail_path, which fetches the local path of a thumbnail and optionally the filetype of the actual thumb (jpeg or png). it needs the new permission
the /request_new_permissions command now accepts a permits_everything bool as a selective alternate to the basic_permissions list
the /verify_access_key command now responds with the name of the access key and the new permits_everything value
the API help is updated for the above
new unit tests test all the above
the Client API version is now 71
client api refactoring:
the main ClientLocalServerResources file has been getting too huge (5,000 lines), so I've moved it and ClientLocalServer to their own api module and broken the Resources file up into core functions, the superclass, and the main verbs
fixed permissions check for /manage_popups/update_popup, which was checking for pages permission rather than popup permission
did a general linting pass of these easier-to-handle files; cleaned up some silly stuff
I added 'you can scan for this' flags for JXL for rotation EXIF, has EXIF, has non-EXIF, has transparency, and has ICC Profile, which we forgot to add when first rolling out JXL support. all existing JXL files will be scheduled for rescan of these properties on update, so if you have any wrongly rotated JXL, they should fix themselves soon after update (issue #1684)
the 'duration' duplicates filter comparison statement now says the +/-% difference in duration. if it is only +1%, it is now clearer
if a server gives a 429 (Too Many Requests) with a 'Retry-After' response header, hydrus now obeys this rather than its default server retry time as set in options->connection (issue #734)
added help->debug->debug modes->macOS window position fix test to test a fix for macOS windows always getting delta y position on init
drag-cancelling tech:
the new drag-cancelling tech was false-positive cancelling fast drags where you moved the mouse outside of the window 200ms after the initial click press. this is fixed
the new drag-cancelling tech no longer occurs on macOS, since it was raising an error. it seems due to system policy you can't programatically cancel a drag on macOS in Qt (issue #1685)
sidecars in export folders:
an export folder with sidecars will now fill in any sidecar gaps on every run. if a file should have a sidecar but currently does not, the sidecar will be generated (issue #1682)
an export folder set to 'synchronise' (which deletes files from the export destination if they are no longer in the export folder's search) will now be more careful not to delete sidecars
some running from source stuff:
for users who run from source, the 'test' PySide6 is updated from 6.8.1.1 to 6.8.2.1, and the qtpy from 2.4.2 to 2.4.3
added a open_venv.ps1 script to easily activate the venv in powershell
duplicates auto-resolution:
my work this upcoming system grinds on. I successfully did the thing I stalled on a couple weeks ago, and now the only big thing I still have to do before advanced users can try it is a preview panel and a billion unit tests
the database modules are live! they boot and will initialise their (as-yet empty) tables on update to v612
the daemon is now live! it runs its full mainloop and consults the database modules for work
rewrote some daemon-db interaction, mostly for KISS and deduplication of responsibility
figured out some tech to reset duplicates rule search progress on edits and user command
simplified the daemon workflow so that rules hold on to cached search counts and report their own search status
fixed some stuff in rule-setting and status fetching
the auto-resolution rule object is now serialisable and could be saved to disk if I enabled the edit UI
rules now report their correct search status, no longer a placeholder
the daemon now reports the correct running status, no longer a placeholder
, go into the UI, and try out the suggested pixel-perfect jpg & png rule. I want to know if--
it all makes sense to you
where it runs smooth
where it runs garbage
any errors?
out of interest, what do you get? Of ~800k potential pairs, I had ~6,000 jpg/png pixel pairs, resulting in ~4,700 total actual processed pairs (numbers shrink because multiple good files can share the same bad file). speed was bleh in the preview viewer (about 30 seconds to load the preview numbers) but nice when doing work: only a second or two to save the rule and then ~20k files/s in the search stage and 10 files/s in the processing stage. about 7 mins to ditch 7.5GB of Clipboard.png, hooray
it is linked in the main help directory, too, under 'advanced'
made it so you can double-click or enter/return any pass/fail test row in the auto-resolution preview panel to open that pair in a normal media viewer
added work on these rules during idle/normal time to the cog button on the auto-resolution sidebar tab
wrote up 'work hard' functionality and wired up the button--however I think I might remove this, since the system works well enough on its own. let me know what you think
reworked the preview panel to have a two-stage search. it spends a whack of CPU time fetching the total count of the search, and then the sample part works faster afterwards and can be hit over and over. it still sucks and I have another idea to speed things up, we'll see
updated the main duplicates auto-resolution maintenance routine to do the search step in blocks of 8192 rather than trying to do everything at once as soon as a rule is added. I hesitated doing this earlier, since the large one-time init search is more efficient, but we are going to be doing incremental searches in normal operation, so making small search chunks work well is not optional. I think the biggest potential weakness of this whole system is going to be incremental searches on new potential pairs when the potential duplicate pairs search of the auto-resolution rule has significant per-search overhead, but which searches will those be? sounds like we'll find out in the coming weeks. thankfully, jpg/png pixel dupes seems to scale excellently, so we are good for today
misc auto-resolution stuff
reduced flicker on the main 'auto-resolution' review panel as it works a real rule
fixed a 'hey I guess the auto-resolution preview panel is 1200px wide now' bug on certain rule-fetch validation errors
renamed 'A will pass' comparison UI labels to 'A will match'
the preview panel lists now scroll one row per scroll (it was doing 3 before, lol, and the default height is ~2 rows)
the auto-resolution preview panel (particularly the 'will be actioned' list, with its third column) should size itself better if you have large thumbnails
if the auto-resolution preview panel had results but the search changes and now we have no resullts, the test result lists will now appropriaterly clear themselves
the preview result lists now generally clear nicely before a new search is started, so if something actually goes wrong, you don't have old test results hanging around
fixed incremental potential duplicates search on non-pixel-duplicate searches
unit tests
wrote unit tests for the MetadataConditional
wrote unit tests for all the predicate types that can work in the MetadataConditional
wrote unit tests for the Selector
wrote unit tests for the one-file Comparator
wrote unit tests for auto-resolution rules: for editing at the db level; for syncing to existing, new, dissolved, and resolved potential duplicate pairs; performing search; performing resolution
misc
fixed a typo bug that broke the maintenance job that resets all potential duplicate pair search. it was related to the recent auto-resolution integration
fixed a typo bug that was stopping the 'review accounts' repository admin panel from opening
added an FAQ about the extensionless files that appear in your file storage if you sync with a repository
added LINUX DEBUG: Do not allow combined setGeometry on mpv window to options->media playback. if you have crashes on X11 in v614 when zooming mpv windows, give it a go and let me know what happens
thanks to a user, added a note to the install help that if you are on Linux & Wayland, adding the WAYLAND_DISPLAY environment variable, which forces the program to run in XWayland, seems to relieve many UI bugs (issue
updated the tag filter to exclude many weird unicode characters. all sorts of Control Characters, right-to-left formatting, zero-width spaces, surrogates, and more is now all removed. Zero-Width characters ZWNJ and ZWJ are allowed unless the rest of the tag is only in extended-latin. the hangul filler character is allowed if the tag includes other hangul syllable or jamo. there is no perfect solution here, but a bunch of mis-parses and other garbage is cleaned up with this (issue #1709)
your client will clean its tags on update. this will take about twenty minutes if you sync with the PTR. you will probably ~30,000+ bad tags--don't worry about it too much, and everything is logged if you are super interested in the saga of ç¿«?ç¸-legacy-decode-jank, ( ?° ?? ?°)-that-includes-a-hidden-Byte-Order-Mark, and a relatively common normaltag[ZWNJ] that's probably either an IME input mistake or some HTML parsing error from years ago
I've been fairly aggressive here, so if I have broken something you do want to input, or something that requires temporary invalid status via an IME, let me know. rest assured, however, that everyone's favourite compound emojis such as ??????????? should still work
linux build:
the Linux build failed last week because I missed notifications about the Ubuntu 20.04 runner being retired. sorry for the trouble! this has happened before, so I am going to keep a better check on retiring runner news in future
the Linux build is now generated on the 22.04 runner. there are some .so file differences, but it seems to all boot ok, and in one case it actually fixed a previously broken mpv load. also, a normal extract-update seems to have worked in our tests, so we don't think a clean install is strictly needed. You might like to do a clean install anyway, just to be neat:
AVIF rendering is fixed. I confidently wrote some code last week that said 'when Pillow updates to 1.21.1, don't load the recently deprecated AVIF support from our external plugin any more because Pillow will now have it natively', and then the Pillow update happened last week and they decided not to bundle in AVIF in their convenient wheel because it bloated their files. I have undone my version check, frozen the plugin at its current version to keep support, and will check this manually on the actual version of Pillow in future before I switch over again. sorry for the trouble! (issue #1714)
added a new Tell original search page to select exit media when closing the media viewer checkbox to options->media viewer (default on). this lets you turn off the behaviour where your exit media is selected in the thumbgrid when you exit a media viewer (issue #1712)
the default value for When maintenance physically deletes files, wait this many ms between each delete: in files and trash is now 600ms, up from 250ms. if you are set to the old default of 250, the update will bump you up. furthermore, the widget in files and trash is now a rich time delta widget rather than just a ms integer spinner
the 'are you sure you want to exit the program, these pages say they are not done' yes/no dialog now spawns with yes disabled for 1.2 seconds, just like the archive/delete filter confirmation, enough to jolt you out of an autopilot enter press
default downloaders:
the safebooru parser is more careful about fetching valid associable source urls. it was previously juxtaposing the safebooru domain with non-https-having garbage
added a thread parser for holotower
deleted some parsers and url classes for long-defunct sites
sidecar sorting:
sidecar objects no longer do a hardcoded sort of their strings before the export step. if you set a different sort via the 'processing' step's string processor, that's directly what will export to the destination
all new sidecar objects now start with a--and all existing sidecar objects will update to get a new--processing step that does 'human text sort (asc)'. thus, all sidecars should continue behaving pretty much as they were before, but if you don't want that, you can now edit it!
actual vs ideal tags:
the right-click tag menu that shows current parents and siblings now shows the ideal tag display space, rather than the actual. the difference between these two is the ideal is what your settings say whereas the actual is what the client currently has calculated as per tags->sibling/parent sync->review current sync. this guy was previously showing the actual calc, which was revealing confusing interim technical states after the preferences changed. I am not sure if this change is correct or helpful and suspect I'll need some better UI around here to quickly detect and explain a discrepancy
updated the /add_tags/get_siblings_and_parents help to discuss that it fetches actual rather than ideal tags
duplicates auto-resolution:
the duplicates auto-resolution UI is all enabled, and I've un-hidden the 'add rule' button. have fun with this new tech, but don't go crazy yet. I think pixel-duplicate pairs are now easy to solve if you have firm preferences about keeping exif etc..
finished off a comparison rule for duplicates auto-resolution that tests things like 'A has more than 2x the num pixels as B'. it can test size, width, height, num_pixels, duration, and num_frames and supports equals, not equals, greater than, less than, approx equal (percentage or absolute), and you can set a coefficient (A has more than 2x filesize of B) and/or an absolute delta (A has more than 200 px more height than B)
added two hardcoded comparison rules for 'A and B have the same/differing filetypes'. nice and simple way to ensure you are or aren't comparing like to like in a rule
the one-file comparators can now do exif, icc profile, and human-readable metadata tests
the 'add suggested rules' button now has three choices--the original pixel-duplicates jpeg/png one, and pixel-perfect pairs - keep EXIF or ICC data (eliminate pixel duplicate pairs of the same filetype where only one has Exif/ICC data) and pixel-perfect pairs - eliminate bloat, (eliminate pixel duplicate pairs of the same filetype where neither have EXIF/ICC data but one is smaller than the other)
the 'action' column in the duplicates auto-resolution preview panels now tooltips to the full text. if this becomes like 100 tags and several URLs, you can now read it!
although we can now poke at the easiest dupe pairs, I think we are still missing a puzzle piece to differentiate alternates from duplicates programatically, even on 'exact-match' searches. we either need cleverer and higher-resolution phashes or a comparison rule that does hardcoded pixel inspection and allows for 'A is > 99.7% pixel-similar to B' in some semantically rich way so we can automatically differentiate jpeg artifacts from banners or watermarks or even colour-only costume changes. while better and perhaps colour-sensitive phashes may come one day, I am going to go for this pixel comparison tech. this will add render CPU cost to each pair decision, which is going to add some bumps to this whole workflow, particularly the preview window, but I suspected we'd have to do this so I've mostly built for it
boring duplicates auto-resolution:
wrote unit tests for the new system predicate media-result-extraction system for the types it can currently extract
wrote unit tests for the new relative pair file info comparator
wrote unit tests for the new hardcoded filetype comparator
wrote unit tests for the new one-file comparator capabilities
the number test widget now emits value-changed signals
fixed an issue where many system pred stubs, until now only appearing temporarily, would not serialise
fixed an issue with the duplicates auto-resolution preview panel where the 'pass' list was not ordering correct for rules beyond the pixel dupe test rule
fixed an issue where the duplicates auto-resolution dialog would not allow you to add more than one new rule per dialog session
brushed up the auto-resolution help a bit
boring cleanup:
moved the 'human text sort' stuff from HydrusData to HydrusText, and improved its sort reliability when strings differ on order of int/str
merged the generic tag sort code into the human text sort system, since they were both doing the same thing. cleaned up some bad old ideas along the way
fixed some bad old tag processing when generating thumbnail banners and the file has more than two tags of a particular namespace and at least one tag includes non-number data
a bunch of places that have some text beside a widget, e.g. the 'show whole chains' checkbox in 'manage tag parents', now copy the tooltip from the widget itself to the text
in services->manage services, all numerical rating services have a new 'icon padding' value, which allows you to squeeze the stars together or push them apart. set to 0 to restore the old 'loading bar' way of presenting (square) numerical ratings
the manage ratings dialog has better layout
also, two new checkboxes in options->gui pages allow you to have the client ask you to auto-name new 'page of pages' on either creation or 'send pages to a new page of pages' events
options:
file->shortcuts is gone! the same panel is now under file->options->shortcuts. I still hate how this guy works and want to merge all the sub-panels into one big user-friendly list with a filter and stuff
the 'custom shortcuts' panel is now viewable in non-advanced-mode
options->sort/collect is now split into tag sort and file sort/collect. the 'namespace grouping sort' is now in the correct location under the tag box whoops
options->media viewer has all hover/background related options sent to the new options->media viewer hovers
options->gui pages has had a reshuffle and the session stuff is pulled out to a new options->gui sessions
all the stuff in options->speed and memory now starts collapsed
the options dialog should be a little shorter by default now
rewangled how the 'replace left/right with primary/secondary' and 'numpad non-number keys are normal' options in shortcuts work in the sub-dialogs--they still appear to apply instantly in the dialogs, but they now reset correctly if you cancel out of the main dialog
the edit shortcuts panel now only writes to disk on ok if you actually edited something
file relationships:
added a new 'media' file relationships shortcut command, file relationships: delete false positives within selection, and an equivalent menu command when you have multiple thumbs selected, delete all false-positive relationships these files' alternate groups have between each other, which removes false positives only between pairs of files in the selection, rather than the other 'remove false positives' command, which removes all known false positives. if you accidentally set a false positive on a pair, this is now specifically undoable by selecting the pair and hitting this
when you do the various 'delete false positives' actions, you now get a popup saying how many were deleted
gave the 'file relationships' file menu a quick pass. it stacks better and now says when a file has no relationships. the reset/remove submenus are also split apart to clarify action importance. this horrible thing should nonetheless absolutely be replaced, wholesale, with a nice review and/or edit panel
cleaned up some of the various duplicate merge logic, replacing hacky stuff with unified calls etc.. I think some redundant potentials are now removed more smartly when you merge certain alternate group members together, and I think the 'confirmed alternates' list is properly cleared of B entries when you merge B into A now. some 'merge alternate groups when two files are set dupe' stuff is clearer and works more thoroughly too. I expect to add a 'clear orphans' maintenance command here in future to handle these legacy issues and cleverly fix any db damage
when you explicitly set files as alternate, any previous false positive relationship between the two is now repealed rather than the job silently failing
duplicates:
the 'A and B are visual duplicates' test now gives a negative result if either file has transparency, confidently if only one does and not if both do--it isn't smart enough to handle this yet
the approve/deny buttons in semi-automatic auto-resolution rule review now do their work off the main thread and no longer block the UI on big jobs
the 'preview' panel of an auto-resolution rule now fetches each block of pairs in a random order it figures out every dialog boot. if you open up a rule that has worked a bunch already, you can sit there watching it climb up to 500,000/700,000 and find nothing since it already did that front work. now it fetches randomly and will start population more swiftly. of course the results will now be in a roughly different order every time you boot the dialog, so that's not amazing--let me know how you find it. I think I may be adding a lot more random access like this, and that means where we will want unified pair sorting settings (much like we want in the manual duplicate filter, which still produces a garbage hardcoded fixed order of pairs!)
subs and downloads:
a core multi-column list update routine is optimised significantly. when many new items come to a list after an edit event (usually mass add or some mass edits), it now selects all items at once. previously it was doing so one at a time, which was triggering a billion button update calls. doing a mass paste into a subscription, or doing a 'separate sub of 1000 queries into 1000 separate subs of one query' now takes about three seconds, instead of many minutes
when you paste queries into a subscription, if some already exist in the sub but are DEAD, you are now yes/no asked if you want to revive them
if a subscription has a gallery page that decides searching should stop, which for this purpose typically means it has 'caught up' to a previous sync, all additional pending gallery url jobs of the same url class in the gallery log will now be set to 'skipped' with a reason of 'previous blah url said: (original stop reason)'. this ensures that downloaders that produce multiple gallery pages in one step are now processed efficiently and obey the catch-up mechanic rather than overspilling the cull limit and adding old files back in--as soon as one page is 'caught up', the rest are skipped. I made this only skip of the same url class so as not to break NGUG subscriptions. if an NGUG produces multiple gallery pages of the same class, they may be thrown off by this since they will no longer check every outstanding gallery url unless the previous produced something useful--if you are clever enough to know what this means, let me know how you get on
when a network job downloads a file from a Post URL, it now registers the bandwidth use to the Post URL's domains as well, if they differ. this should soon fix the issue where a site that produces files on completely different domain was not pausing when the file domain hits its bandwidth rules (file urls override bandwidth rules within three seconds in order to keep post/file url hits close together, which helps some token stuff, but if the post url never becomes aware of how much bandwidth it is eating, it just keeps on chugging. now it won't!)
added a checkbox to stop the 'override bandwidth in 3 seconds' rule under options->downloading, but I'm hoping the above overall fixes the issue
misc:
the page tab menu gets a new 'collapse pages' submenu. depending on where you click, it'll say 'this page', 'pages from here to the right', and 'pages to the right'. it will suck up all the files from the given pages and place them in a new locked search page, closing the old pages in the same step. use this for when you have eight small import pages in a row you want to collapse into a single processing page. it isn wrapped in a yes/no. it works on 'page of pages' too. this action is cool to do, so try it out
the 'send down to a new page of pages' gets the same 'pages from here to the right' option too, so you can do both operations from the very left-hand side if that's what you want
fixed a benign error popup when you hold the 'next image' shortcut while closing out on the last pair in a duplicates filter
if attempting to send a file to the OS recycle bin results in a 0x80270021 error (many causes for this, but it seems it can also be an unluckily-timed sharing violation or a recycle bin with 100k+ items in it and being mega laggy), it is now re-attempted twice, with short breaks in between
when, under services->manage services, you open an edit panel for a rating service that has stars, the client now repopulates the cache it stores of all your current custom svgs. no need to reboot now, just reload the dialog and it'll show what is current
I may have solved some indirect threading deadlocks. all threaded jobs also initialise just a tiny bit faster now
blurhash generation, which can be a whack of CPU per import, is now a tiny bit more efficient
boring stuff:
removed setuptools from the requirements.txts that don't have pyinstaller. we freeze the setuptools version to ensure a stable frozen build structure, so this isn't explicitly needed anywhere else
refactored some of my advanced dev requirements.txts
brushed up the 'running from source' help doc and added some info about testing/installing pip and venv, which aren't always available by default in Linux
replaced all the typing.Collection style typedefs across the program with the collections.abc equivalents. this stuff was deprecated starting Python 3.9 lol. also, I'm pretty sure this was already true, but hydrus will now definitely not run in Python 3.8
replaced all the basic typing.List style typedefs across the program with the standard list equivalents. same deal as above
overhauled the 'db duplicates' unit tests, breaking up the previous monolith rollercoaster into something a bit more atomic. it is still a huge mess, but much better than before
added some unit tests for the new 'remove false positives within selection'
Witaj gościu. Dziękujemy, że przeglądasz nasze forum ale czy wiesz, że zakładając konto możesz w pełni korzystać z dobrodziejstw jakimi są promocje i konkursy. Nie zobaczysz tu też żadnych reklam a rejestracja zajmie Ci tylko chwilę.
Ta strona wykorzystuje ciasteczka (cookies) w celu: utrzymania sesji zalogowanego Użytkownika, gromadzenia informacji związanych z korzystaniem z serwisu, ułatwienia Użytkownikom korzystania z niego, dopasowania treści wyświetlanych Użytkownikowi oraz tworzenia statystyk oglądalności czy efektywności publikowanych reklam.Użytkownik ma możliwość skonfigurowania ustawień cookies za pomocą ustawień swojej przeglądarki internetowej. Użytkownik wyraża zgodę na używanie i wykorzystywanie cookies oraz ma możliwość wyłączenia cookies za pomocą ustawień swojej przeglądarki internetowej.
