/t/ - Technology

Discussion of Technology

Index Catalog Archive Bottom Refresh
Options
Subject
Message

Max message length: 12000

files

Max file size: 32.00 MB

Total max file size: 50.00 MB

Max files: 5

Supported file types: GIF, JPG, PNG, WebM, OGG, and more

E-mail
Password

(used to delete files and posts)

Misc

Remember to follow the Rules

The backup domains are located at 8chan.se and 8chan.cc. TOR access can be found here, or you can access the TOR portal from the clearnet at Redchannit 3.0.

RIP David Lynch

/vhs/ David Lynch Marathon


8chan.moe is a hobby project with no affiliation whatsoever to the administration of any other "8chan" site, past or present.

You may also be interested in: AI

(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #11 Anonymous Board volunteer 02/05/2025 (Wed) 22:34:45 No. 17183
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Users can choose to download and share tags through a Public Tag Repository that now has more than 2 billion tag mappings, and advanced users may set up their own repositories just for themselves and friends. Everything is free and privacy is the first concern. Releases are available for Windows, Linux, and macOS, and it is now easy to run the program straight from source. I am the hydrus developer. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. Past hydrus imageboard discussion, and these generals as they hit the post limit, are being archived at >>>/hydrus/ . Hydrus is a powerful and complicated program, and it is not for everyone. If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/
https://www.youtube.com/watch?v=Xqg2KSdBLRU windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v608/Hydrus.Network.608.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v608/Hydrus.Network.608.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v608/Hydrus.Network.608.-.macOS.-.App.dmg linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v608/Hydrus.Network.608.-.Linux.-.Executable.tar.zst I had a good simple week. Just a bunch of small fixes and improvements. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights The new mpv error hook that detects the 'too many events' error is overhauled. It shouldn't fire unless there is a serious problem now. If you were hit with false positives, sorry for the inconvenience the past few weeks! After the recent successful 'group namespaces (user)' update in tag sort, and after talking with some users, I have decided to make that namespace grouping always sort the same way, no matter which way you sort the underlying groups of tags. This means, e.g., that 'creator' tags will stay at the top, then 'series' and 'character' etc.., whether you say 'most first' or 'fewest first'. See how you like it--I think it is more intuitive now. I improved how the multi-column lists across the program set their minimum and initial height, and in most cases and styles they appear to be pixel-perfect to the number of rows desired. Particularly, in the gallery and watcher downloader pages, where the multi-column lists dynamically size themselves based on how many items they are holding, it should fit exactly. I find the perfection a little eerie, so I may add just a few pixels of padding so the back of your brain knows there isn't anything to scroll down to. next week I finished the duplicates auto-resolution database work this week, so all that's left is a daemon to run everything and a preview panel. I'll see if I can get the daemon done.
just wanted to say: keep up the great work!
(51.36 KB 540x540 borg.png)

Just wanted to say: hi
Would it be possible to add an undo to pages? I sometimes think I have my browser selected and press F5, only to wipe a page on hydrus because the search is empty and I've been manually dragging files to the page. A searchless page could also work.
>>17188 This used to happen to me all the time, so I just deleted the shortcut.
>>17189 Not a bad idea.
>>17186 >>17187 Thanks lads, keep on pushing. >>17188 Yeah, I've done this myself. I have a barebones undo system, but I never wrote the original 'do a thing to a page' pipeline to plug into it correctly. Ideally, you want this chain recorded of all changes, and all changes tell the program what they did and how to undo them, but stuff like 'remove files' command just gets swallowed right now. I've slowly been converting my internal objects to better support this, but it'll be a while yet before I can support anything I'm happy with. On the other hand, I'll be more immediately working on some better tools to 'sync' a page to its current results. If you are a long-time-enough user, you'll remember that 'open these files' pages used to have no search. They wouldn't respond to F5 either. I replaced them with a page that spawns with a system:hashes search, for KISS reasons, and while you can flip the 'searching immediately' button to reduce problems, we are now broadly missing the ability to have a 'read only' page that just holds some files mate. I'll be figuring out a checkbox or something that freezes or pins a page as it is so guys like you don't lose their work on a scratchpad page.
In the duplicate filter, when you compare videos, it seems to always give the longer video a much higher score. it seems to give it around 50 points, with no way to change it. this happens even if the "longer" video is a single frame longer. this is probably a bug
Is there an option somewhere, or can we get it by default, that you get a notification for when cookies that are necessary for certain downloaders expire? >>17192 >keep on pushing. I will.
I had a good week. I mostly did boring cleanup work, but by serendipity I figured out a set of fun new file sorts that work using a file's average colour, so you can now sort from lightest to darkest, grey to saturated, and by hue. The release should be as normal tomorrow. >>17194 Thanks, I fixed it for tomorrow to do 1-50 depending on the relative lengths, capping at twice the length. That's probably still a bit wrong, so let me know how it feels and we'll keep adjusting. >>17198 The login system can kind of do this, except when a cookies is expired/missing, it then wants to start a login process. I'm not sure you can set up a login script that recognises a missing cookie but then won't cause domain errors when its empty or whatever login process inevitably fails. Maybe it is ok if it fails, since jobs on that domain will probably just cycle on 'login problem' delays until the cookie is filled in or you re-validate the login script for another attempt. Bit bleh all round though. Maybe I should consider starting an overhaul of that system, which I haven't touched in ages and is really starting to creak. If you use Hydrus Companion, it can regularly sync your cookies and you avoid these issues, but if you have to do it manually, in your situation, I think you might have to monitor it manually too. Maybe you can resync every two weeks or something. Sorry!
>>17200 >Maybe I should consider starting an overhaul of that system, which I haven't touched in ages and is really starting to creak. It's not a huge issue for me. Most of my cookies last for years, and the nearest expiring one is 4 months away.
>>17200 >grey to saturated, and by hue. This is immensely useful for tagging things as "black and white/monochrome/greyscale" or whatever tag one might use for that. It may also be able to sniff out things in other color scales for easy tagging. An image that's entirely in shades of blue is almost as lacking in color as an image in black and white.
Can I do a search of files that are only in multiple domains. Under 'edit multiple locations' it seems that I can do 'domain A' OR 'domain B', but not 'domain A' AND 'domain B'.
>>17203 Not sure what you really want, but it seems you want either 1. Search files that for example must be in domain A AND also in domain B. For that you should change your domain to one with a broad scope, for example 'all my files' or better 'all local files' (Advanced Mode), which would also contain trash. Then do several searches with 'system:file service: system:is currently in my files system:is currently in test location That way it searches for all files that are in 'my files' AND 'test location' at the same time. 2. You want to activate the checkboxes in 'edit multiple locations' to take a look into those domains at the same time and think you can't. Hydrus let's you activate the boxes that make sense. If you are not in Advanced Mode, you might have 'my files', 'all my files', 'trash' and maybe you added a new one, say 'test location'. If you check 'all my files', you can't check 'my files' or 'test location', because 'all my files' already contains them. You can check 'all my files' + 'trash' though at the same time, because 'trash' is not included in 'all my files'. In Advanced Mode there are much more. If you check 'all local files', you can't check all mentioned above, because it contains them all. You can for example check 'my files' + 'test location' + 'trash' at the same time, because you didn't check a parent that contains them already like 'all my files' or 'all local files'. Also this might help: https://hydrusnetwork.github.io/hydrus/advanced_multiple_local_file_services.html
https://www.youtube.com/watch?v=A1QUUmShhMM windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v609/Hydrus.Network.609.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v609/Hydrus.Network.609.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v609/Hydrus.Network.609.-.macOS.-.App.zip linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v609/Hydrus.Network.609.-.Linux.-.Executable.tar.zst I had a good week. I mostly did boring cleanup work, but there's some neat new colour-based file sorting to try out. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html macOS zip First off, the macOS release is now in a zip. We've had technical problems with DMG creation recently, and zip is much simpler and faster and more reliable. You'll now be double-clicking the zip to extract the App and then dragging it to your Applications directory yourself. I am recommending all macOS users consider running from source, since it skips a bunch of App hassle, and particularly so if you are on Silicon (the App is intel atm, which causes a very slow load time on Silicon). https://hydrusnetwork.github.io/hydrus/running_from_source.html colour sorting A user mentioned that the 'blurhash' we recently added stores a file's average colour. I played around with the idea and figured out some fast 'sort files by colour' tech. It works for all files with thumbnails. They are all under a new 'average colour' menu, and there are five types: 'lightness', which obviously does dark to light; 'hue', which orders files according to the rainbow colour wheel you will have seen in any HSL colour palette panel; 'chromatic magnitude', which is basically saturation and does grey to colourful; 'balance - blue-yellow', which uses the idea of 'chromaticity' and scales from the bluest files to the most yellow; and 'balance - green-red', which similarly does greens to reds. This isn't perfect, since it works on average colour, so a file with a very bright red blob on a dark background might be counted the same as a brown mush, but it is cool and fun to play with, and it is a good base for future, cleverer colour histogram tech. I'm really happy with how it turned out. Give it a go and let me know how it works for you. next week I did a bit of duplicates auto-resolution work this week, but I wasn't super successful. I'm going to do a bit more UI so I can get a better idea of how I want the daemon to work.
I made a new pixiv subscription and it failed to get any 18+ files, so I sent the logged in cookies via Hydrus Companion, tested that I could grab 18+ files by sending a tab via Hydrus Companion, and that worked, but deleting and remaking the subscription just gave me the same single SFW file, no 18+. What am I doing wrong?
>>17206 Figured it out. Had this issue before. Cookies don't stick right when sent from an incognito tab.
Some minor issues with the manage notes window that make it hard to use: >Whitespace at the start or end of the note is removed. Whitespace is also removed from the start or end of each individual line. Also, if note has multiple newlines in a row, even in the middle of the note, they will be turned into just one upon pressing apply. I want to use spaces to indent and I want to be able to have multiple newlines in order to organize information nicely... >If a file has multiple notes and you have the option "Start editing notes with the text cursor at the end of the document" unchecked, it only works for the initial note that was opened. Switching to another note tab will put you at the end of the document instead of the start. >It would be nice if when you pressed escape or cancel to leave the notes popup and you've made edits it would warn you like "unsaved changes will be lost. continue?" sort of thing. Please take a look. Thanks.
(1.26 MB 3000x2500 cuter.png)

>>17205 >colour sorting Thanks, this is useful indeed.
Speaking of notes, hydev, could we get a mass edit option for notes? Or a least a mass delete?
>>17204 >1. Search files that for example must be in domain A AND also in domain B. > >For that you should change your domain to one with a broad scope, for example 'all my files' or better 'all local files' (Advanced Mode), which would also contain trash. Then do several searches with 'system:file service: > >system:is currently in my files >system:is currently in test location > >That way it searches for all files that are in 'my files' AND 'test location' at the same time. This is what I wanted, thank you. Trying to find files that are in BOTH file domains, so I can remove them from the domain they do not belong in.
>>17208 >Whitespace This is a tricky one. The notes import workflow relies heavily on a bunch of merge tech, and it is really useful for me to trim and collapse excess whitespace so notes from different sources can be compared well. That said, all this merge tech worked out pretty shit tbh and I hate it, so it wouldn't kill me just to make the text trimming smarter and add some options so users can do what they want. I'll see what I can do! >bad option Thanks, I will fix it! >warn on cancel Great idea! >>17213 This is doubly tricky, since the UI gets suddenly complicated and I need to do (even more!) merge tech to show how the files share notes and stuff, but I'll see what if I can make it work like the multifile URLs editing dialog, where it has some red text or something saying 'hey, careful, you are editing for a bunch of files here, all will be affected.' Some way to say 'add this note to the files that don't have it yet' would also be nice.
Are there any plans to make it so you can click and drag to select multiple files?
I just installed Hydrus in MX Linux and the MPV viewer won't load. The library libmpv2 is installed and even I upgraded it with the version in the Test Repository but I had not luck. Anyway, I'm just posting to inform devanon what and where MPV is failing, as I switched to hydrus from source and it is working great, see pic 4.
Just checking in to see how much of a crack dream reverse PTR + IPFS is With booru enshittification and the repeated attempts at making porn illegal I feel something like it may become necessary
>>17224 I'm no Linux expert, but I've seen that 'g_module_open_full' thing before--hit the 'Linux' tab here and scroll down a bit: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#installing Looks like you will be adding some .so files to your Linux install. I guess it is something to do with Ubuntu not agreeing with your Linux flavour about where some things should be. Failing that, running from source can help with these weird dll load problems since you'll then be completely native. >>17222 Yes, I would like this. I do it myself by accident sometimes, forgetting I still haven't added it. My thumbnail grid is waiting an important overhaul, after which I expect it will be much easier to insert this sort of behaviour. >>17225 IPFS use a slightly tricky hash/identifier for files--iirc it is a bit like a torrent magnet, in which all the blocks of the content have separate hashes, and the hash of the master file that lists those blocks is the main 'id'--and, as I understand, you just cannot predict an IPFS hash with only an SHA256 hash. And I'm not up to date, but I understand you cannot query the IPFS network using a hash like an md5 or sha256. So, while it would be possible to do this, you'd also need a way to propagate (SHA256-IPFS multihash) pairs so our hydrus db would be able to cross-reference tags and stuff to an IPFS multihash that would get you the file (so you could, say, do a search for files you don't have but which have the tag 'samus aran' on the PTR and have a known IPFS multihash, and that list could spawn a new downloader that would work pretty much like a booru gallery search). I've thought about this, and I generally call it a PUR for Public URL Repo, or basically a 'hey this repo stores known locations/ids for SHA256 hashes', for which IPFS multihash could be one mapping type we could add. Writing a whole new Repo would be a ton of work, and I am still of the mind that managing files remains the trickier problem to solve than sharing them, but I agree we've seen some more restrictions recently. I'm mostly concerned with the big CDNs putting anti-downloader captchas on many sites. And I'm worried AI is going to break captcha and make Real ID the standard for anything with a web domain. We'll see how it goes, but yes, I think there could be a way forward here if we wanted to. The Client API, also, is much more mature these days and is open to someone who had an interesting idea. I'm toying with the idea of client-to-client comms in the back of my mind, and I wouldn't be surprised if I slowly pivot over the coming years to a cleverer client, likely with AI tech to solve common problems like tagging character names and simple nouns, and retiring or minimising the server. We'll see how everything shakes out.
I had an ok week. I ended up mostly working on some boring cleanup and refactoring, so there's not much exciting to talk about. There's new file viewing statistics stuff and some misc quality of life. The release should be as normal tomorrow.
Hello! I'm having an issue with a custom parser. In cases when it tries setting two notes with different names but the same content, only one of them gets associated to the entry. Test parse always shows them correctly and separately, and they both always get associated properly when they differ in content. Could it be a bug?
>>17228 I was talking with another guy, I think by email, about this exact same issue this week, but we resolved it, so I presume you aren't him. My note merging logic isn't something I'm totally happy with, but it has a bunch of cases where it says stuff like 'if we are adding a name+text, and the name already exists and the note that comes with it is a prefix of the new text, then replace the existing text' in order to stop it adding name(1)+text duplicates. A similar thing basically says 'if the text we want to add exists anywhere, then don't add it, since it'll just be a dupe'. This stops you getting the same gumpf for the 'danbooru artist comments', 'gelbooru artist comments', 'e621 artist comments' etc... It isn't super beautiful, but it saves some spam. So, it isn't a bug. That said, I don't like it! And when the two duplicate note texts come in the same parse job, it pseudorandomly selects which will be the first. I've got several ideas and heard several user opinions on how notes should change going forward, some radical, but I am not sure what I want. How would you, in an ideal world, like to handle the overall problem of duplicate note names and/or texts?
https://www.youtube.com/watch?v=lUVCJGSTcO8 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v610/Hydrus.Network.610.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v610/Hydrus.Network.610.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v610/Hydrus.Network.610.-.macOS.-.App.zip linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v610/Hydrus.Network.610.-.Linux.-.Executable.tar.zst I had an ok week mostly doing boring cleanup. There's some quality of life improvements. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights The new 'sort by hue' file sort puts anything that is a very grey average colour at the bottom of the sort. The 'edit notes' dialog has some nicer focus logic, and it will warn you on cancel if you have made changes. The menu settings under options->file viewing statistics are simplified. What was previously five different ways to show viewing stats in the file right-click menu is collapsed to two, and there's now a checkbox list of which view types you are interested in showing. The new 'client api' view type is integrated into the UI, and some views/viewtime system predicate parsing is easier to work with. I overhauled the parsing pipeline this week, and I fixed some weird parsing bugs and bad test UI presentation and so on. If you are an advanced downloader maker, let me know how things go. next week I got mired in cleanup this week and did not get to several things I wanted. I'll keep working my immediate small job todo and try to clear things out a bit.
hydrus that allows hardlinks/symlinks instead of copying files when? i don't care if the database breaks sometimes damnit
I think there is a bug when i 'sort by tags: number of tags'. So when i match the domain i am looking into in the search pane (the one under the OR button) to the 'tags' button domain (next to collect/leave unmatched button), let's say both are changed to 'downloader tags' and 'ascending', then the sorting is correct. I suppose you have to match them, because that way the tags you see are the tags you sort exactly. The first files have no tags, then when i hold the right arrow key, i can see how the tag list gets more and more populated with tags and the scroll bar in the tag list gets smaller and smaller. That is how it should be. But when i match the domain of the said buttons to everything else except 'downloader tags' (-> my tags, PTR, all known tags), the sorting doesn't work anymore and is all over the place. In average it might get more and more, but from file to file the number goes up and down. Now i realized how to fix it though. If i change 'sort by tags: number of tags' to a namespace sorting, a second 'tags' button appears right above the first 'tags' button. This is actually the one that changes the behaviour in the 'sort by tags: number of tags' sorting (which is not visible there), that means 'downloader tags' was activated here, so it coincidentally worked in 'sort by tags:number of tags', which i described in the first paragraph. So we would need THIS 'tags' button there instead of the one which is there, which seems it doesnt do anything in terms of sorting by number of tags. I think this 'tags' sort button is linked to the collection buttons left of it, so this changes the sorting of 'collecty by' collections. In short: how to sort by number of tags for 'all known tags'? If it works for you like for me in paragraph 2, then try another domain. Does it still?
>>17183 >weekly releases How do you (or others) keep the flame of motivation alive?
I just realized I had closed my page with many watchers 3 days ago. - restoring a db backup would lose my changes - exporting all files downloaded or modified is very complex and would not necessarily save everything - "last session" backups happened every 5 minutes, so there were not enough of them - apparently sessions cannot be exported - I couldn't find where they are stored in the databases. Not sure I would have been able to use it anyway. - the nearest "exit session" was 6 days old. I chose this option.


Forms
Delete
Report
Quick Reply