Warning: this post is mostly technical, since nearly nothing changed feature-wise. Testing is really needed though, internal changes are quite heavy!
You may find the builds at the usual place: http://download.blender.org/ftp/mont29/
From the “outside”, user point of view, this build does not differ much from previous one from January. Only changes (outside of master work, like preview size) are:
- Some data was removed from the detailed view (modes and owner – UNIX-only, and Blender is not an OS file browser!).
- Columns of compact (default) and detailed views are now fixed-size (among four choices, like for previews view). This is due to internal changes detailed below.
Internal changes were mostly done for future asset engines, but they also improve regular file browser experience, in a nutshell:
- Smaller memory footprint when browsing directories with huge number of items (several thousands and more) – especially in case of preview display mode.
- Much quicker display of those previews.
Plan is to merge this branch (or as much as possible of it) into master after 2.75 release.
Now for the technical details!
The main changes are under the hood – a full rewrite of our file listing code, to reduce memory usage and global computing effort. The idea is to keep as few as possible data from actual directory(-ies) listing, and to generate the full stuff needed to draw in filebrowser window only when needed, following the “sliding window” principle.
So, let’s say your file browser is currently centered on item 4000 in a directory containing 10000 files. We do store minimal data for those 10000 files (what’s needed for sorting and filtering), but only generate complete info (detailed types, strings for size, previews, etc.) for items through 3489 to 4512. Then, if user scrolls slightly to center item 4500, we only trash items up to 3988, and only have to generate those from 4513 to 5012… As a bonus, this means e.g. previews are generated for visible areas first, instead of ‘flat’ top-to-bottom process which would sometimes takes several tens of seconds to reach the bottom of the listing.
This allowed us to reduce our `direntry` struct size (it now basically wraps path and stat info). But the main improvement is with previews – since a few days previews are now 256*256 pictures in master. For a directory with 10000 images in preview mode, that means 2.6GiB of ram – just for previews! While with a sliding window of about one thousand items, we can limit this to a maximum of 266MiB – should we be browsing a directory with tens of thousands of items.
This is a rather extreme example of course, but not uncommon (think e.g. to render directories, or one with one or two hundreds of library .blend files read in new ‘flat’ mode…). And there is another reason why this change is needed: future asset engines. One can easily imagine on-line asset repositories with tens of thousands of items, you just cannot generate in Blender items for all of them at once! With that new code, asset engine only gives Blender the total number of entries, and then Blender requests for a limited range of those as needed (note that ordering and filtering will obviously also be deferred to asset engines).
.blend file items preview has also been moved into IMB_thumb area. This was mandatory, as we do not store previews for all items, we could not read them anymore while doing the initial listing. The main drawback is that it means on first run, .blend files will be read twice. However, there are several great advantages:
- Consistency: .blend library items previews are now handled as any other (image, font, .blend file itself…) – this also means other areas of the code can easily get previews for them.
- Performances: since they are handled by IMB_thumb, they also use the thumbnail caching system – in other words, .blend file is read once, regenerating the thumbnail later will only be a matter of reading the cached .png file!
Also, one side effect of those heavy changes is that previews should be generated much more quickly, since they now use a much lighter `BLI_task`-based threading, instead of the complex and heavy `Job`-based one.
), do not expect (too much) issues on this platform, but you never know with Windows. Cheers, and hope that new year will be full of good things for Blender and all of you!
mont29 