Serendipity Forums

One other question: Why is there two seperate variables for the path to the 'uploads' directory: $serendipity['uploadHTTPPath'] and $serendipity['uploadPath'] ?

Hi Martin!

Well, dropping the serendipity_images table is a thing we will not be considering for the Serendipity Core repository.

The reason is performance and integration, mostly. Always going via file system calls over all files is a pain in the but, so a database really makes sense. It is also much better and easier for future usage cases, when image references are made to a DB ID instead of a filename.

This then enables you to rename an image without breaking referencing in postings. This wouldn't be possible without a database.

Also, storing comments to an image wouldn't work without a database. And you already mentioned hotlinks, which also won't work without a DB.

So maybe you could look out for a solution that implements what you need, without breaking or getting rid of serendipity_images plus the function calls depending on it? Smile

How does a database allow this? In the post, is it not stored as as a reference to the file location, ie 'uploads/pictures/this.jpg'?

It doesn't need to be removed, but maybe it's not strictly necessary unless features like comments and hotlinking are used? It does replicate alot of information that is accessible via the file system, and replication of information is always a maintainence nightmare.

There once was a plugin (that was lost) which did NOT insert the image file reference into the post. Instead it returned "<!--image:4711-->" or some similar code. This was later transformed by querying the database (once, then it was cached). So if you renamed a file later on, the ID would stay the same and the code could get reparsed.

I would really recommend that the defualt serendipity functions use the serendipity_images table. It is much safer for forward compatibility and enhancements, and it takes up less performance because mime-types, author-permissions and many other meta-data can be stored there. So this is not really replication, it is just a level of meta-data cache and storage facility.

 
 `id` int(11) NOT NULL auto_increment,
  `name` varchar(255) collate utf8_unicode_ci NOT NULL default '',
  `extension` varchar(5) collate utf8_unicode_ci NOT NULL default '',
  `mime` varchar(255) collate utf8_unicode_ci NOT NULL default '',
  `size` int(11) NOT NULL default '0',
  `dimensions_width` int(11) NOT NULL default '0',
  `dimensions_height` int(11) NOT NULL default '0',
  `date` int(11) NOT NULL default '0',
  `thumbnail_name` varchar(255) collate utf8_unicode_ci NOT NULL default '',
  `authorid` int(11) default '0',
  `path` text collate utf8_unicode_ci,
  `hotlink` int(1) default NULL,

Plus, the functionality you want to achieve should perfectly work without all these core modifications. It's perfectly fine if your plugin does not require the serendipity_images table - but then you'd need to wrap your plugin hook calls around the existing framework, instead of generalizing your plugin's needs into the core. Smile

Okay, but this functionality doesn't currently exist in s9y right?

And if you renamed the file in the file system, you'd still have to update it in the database table. And if you were going to update it in the database table using some s9y front end, you could have that update the direct <a href>'s in the entries anyway.

Of those entries, all of the information is on the filesystem except the authorid and the hotlink.

Yes, I can implement this stuff as a plugin by only adding hooks for a couple of new events, but I think that by removing the data replication, it becomes easier to manage the s9y media gallery.

And if you renamed the file in the file system, you'd still have to update it in the database table. And if you were going to update it in the database table using some s9y front end, you could have that update the direct <a href>'s in the entries anyway.

The rename functioanlity currently already replaces both the filename and
the database column for the filename.

That's right. And that's two pieces of information you can't easily store within the file. Plus, the DB holds all that as cached information. No need to get the mime type time and again, that would be very redundant.

Again, it's not replication - it is caching plus metadata-storage. You'll block your way for future enhancements by going to the filesystem. And you'll decrease your performance with a filesystem meta-data storage.

Plus, some plugins require this table - so why break it? It would be a step backwards...

(I don't mean you any harm, so I hope you don't mind my arguments Smile

And what if I rename the file outside of s9y? Or delete it ? Or move it?

Yes, we've agreed that we need a table to store the authorid and hotlink, although I suspect that you could store the authorid by directory rather than by file and make it alot easier to work with.

With the mime-type, you can use lazy initialisation to determine it as and when you need it.

And you can be sure that the mime-type you end up with is correct and up to date-> if someone had replaced 'foo.jpg' with a text file, your database would be incorrect.

Further, the performance of checking the mime type of a file has to be significantly less than a database query to determine the same (but possibly stale) information.

Again, it's not replication - it is caching plus metadata-storage. You'll block your way for future enhancements by going to the filesystem. And you'll decrease your performance with a filesystem meta-data storage.

I disagree. You must admit that 9 of the 11 fields of the table are duplicated and can contain stale information.

Future enhancements can use whatever solution is most appropriate for handling hotlinks + authorids. The performance of a database query for the data on a single image is going to be vastly slower than querying the filesystem for the same information.

Don't they all access it via the functions_images.inc.php? They should notice no difference in functionality unless they directly access the serendipity_images table.

Not at all! A bit of healthy debate never hurt anyone

And what if I rename the file outside of s9y? Or delete it ? Or move it?

This is not intended operation. You cannot expect serendipity to fix your links to renamed files if you don't use s9y to rename your files.

You say that so easily, when it fact it isn't. You don't know when you need to "lazily" initialize it...

Why would someone replace the internal s9y media files with other files, without the use of s9y? It's the s9y media database where people are operating in, so they need to use s9y or a plugin to modify content there. They can use FTP upload and synchronizing of course. This is like asking "why does windows not boot up, if I copy a PNG image into it's swapfile?" Smile

Further, the performance of checking the mime type of a file has to be significantly less than a database query to determine the same (but possibly stale) information.

No, it's not, because the DB handle returns all the information at once. So it's performance is always realted to a single DB query call.

Getting 100 Mimetypes is only a single DB call vs. 100 filesystem calls.

Again, it's not replication - it is caching plus metadata-storage. You'll block your way for future enhancements by going to the filesystem. And you'll decrease your performance with a filesystem meta-data storage.

When opening the media database, it mostly queries more than one image. When a blog page renders with 15 images, a plugin that uses the database could stack up those request in one single DB call, which would be faster than 15 filesystem calls.

Don't they all access it via the functions_images.inc.php? They should notice no difference in functionality unless they directly access the serendipity_images table.

I think there are plugins that directly query and join serendipity_images tables.

It still might happen. The concept of allowing easy insertion of images into blog entries is an excellent feature. Forcing the user to only manipulate those images through s9y is not.

To support this feature, a better solution is a plugin which has something like 'update link', which would allow the user to change the contents of an arbitrary <a href="thing"> to <a href="newthing">. That would support the updating of any kind of link, including images.

You say that so easily, when it fact it isn't. You don't know when you need to "lazily" initialize it...

As far as I can see the only time that you need to know this information is if you are calling 'serendipity_fetchImagesFromDatabase' or 'serendipity_fetchImageFromDatabase'. Even in these cases, you probably don't need the mime-type each time, and can use the filename to query for it should you actually need it.

But why force people to keep an 'internal s9y' copy of their media?

Getting 100 mimetypes is the worst case scenario. I timed how long it takes to do it via file systems calls and in a DB call and this is the result:

When opening the media database, it mostly queries more than one image. When a blog page renders with 15 images, a plugin that uses the database could stack up those request in one single DB call, which would be faster than 15 filesystem calls.

The blog will render by using <a href=".."> to make the HTTP client download the images, the database is not used in this case.

I will re-write the filesystem method to optimise the number of calls to the file system, and post some new performance data when I have it.

We don't force the user, because we provide API functions.

If you maintain images via a Windows Application and then use Word to edit the database file, you also don'T expect the Windows Application to work properly after that Wink

But that might not properly work because there are all kind of Regular Expressions where one inserted an image on another server that he doesn't want to change.

Parsing this raw HTML is much harder than using a DB-reference markup.

That's right, but those calls are always required when browsing the media gallery.

It's not a copy, it's a cached representation. That is much different, because it allows to be updated in single- or eval dual-ways.

We/You can add functionality to make s9y be able to import other directories into that storage; but the internal storage should use the Database for storage of Meta-Data and fast access.

Well, 100 is a bit too much, but usually we display 24 images or so per page. That would mean 24 FS calls.

When opening the media database, it mostly queries more than one image. When a blog page renders with 15 images, a plugin that uses the database could stack up those request in one single DB call, which would be faster than 15 filesystem calls.

I still refuse to drop serendipity_images as the preferred storage mechanism because of the reasons already mentioned. Using the FS to always fetch file data is a dead-end and not enhancible or a wise thing to do, when we already have an established database storage.

1. Always have up to date information on the directory contents and the file meta data.
2. New images can be added simply by copying them to the appropriate directory.
3. Performance is the same or better.