Bug in Archive plugin

ads · Post by **ads** » Fri Sep 19, 2008 12:43 pm

Hello,

If i enable the "show entries per category" setting in the archive sidebar plugin then Serendipity shows an odd posting number. In sum i have 7 postings in this blog but S9y outputs:
"September 2008 (12)"
That's wrong.

Here's the query from serendipity_fetchEntries():

Code: Select all

SELECT count(e.id) AS orderkey


                FROM
                    serendipity_entries AS e
                    LEFT JOIN serendipity_entrycat ec
                        ON e.id = ec.entryid
                    LEFT JOIN serendipity_category c
                        ON ec.categoryid = c.categoryid
                     LEFT JOIN serendipity_authorgroups AS acl_a
                                   ON acl_a.authorid = 1
                            LEFT JOIN serendipity_access AS acl_acc
                                   ON (    acl_acc.artifact_mode = 'read'
                                       AND acl_acc.artifact_type = 'category'
                                       AND acl_acc.artifact_id   = c.categoryid
                                      )
                     WHERE e.timestamp >= 1220220000 AND e.timestamp <= 1222811999 AND isdraft = 'false' AND e.timestamp <= 1221819900 AND     (
                                 c.categoryid IS NULL
                                 OR ( acl_acc.groupid = acl_a.groupid OR acl_acc.groupid = 0)
                                 OR ( acl_acc.artifact_id IS NULL

                                    )
                               )

            ORDER BY timestamp DESC

This serendipity_fetchEntries() function does too much magic, i can't find out what's the source of the problem.

Post by **garvinhicking** » Fri Sep 19, 2008 12:54 pm

Hi!

Serendipity cannot properly count entries that are associated into multiple categories. The count is increased for postings for each category it is in.

I also don't know the SQL magic required to properly count entries, this has been a glitch since the beginning. Much of the SQL is needed to properly evaluate read/write permissions to categories, and only by stripping that the count would be alright - but then, the permissions would not be evaluated.

Win some, loose some. IMHO it's better to have article counts counting the categories multiple times than to have a count that includes entries you are not allowed to read...

Regards,
Garvin

ads · Post by **ads** » Fri Sep 19, 2008 1:09 pm

Hae? You say this count is wrong and it has never counted the correct posting number if a posting is assigned to more than one category?

If you can't count it right, don't count it at all.

Sorry, posting a wrong posting count is just plain wrong and this "feature" should be deactivated.

Post by **garvinhicking** » Fri Sep 19, 2008 1:36 pm

Hi!

If you can't count it right, don't count it at all.

The counting is optional and not enabled by default (since it's also a huge performance impact), and some people that do not assign entries to multiple categories can still use the counting feature.

Sorry, posting a wrong posting count is just plain wrong and this "feature" should be deactivated.

No, ultimately it needs to be fixed by a developer who knows a solution. Since Serendipity is OpenSource, everyone to whom this bug is important enough can contribute a fix.

Regards,
Garvin

ads · Post by **ads** » Fri Sep 19, 2008 2:05 pm

garvinhicking wrote:
If you can't count it right, don't count it at all.
The counting is optional and not enabled by default (since it's also a huge performance impact), and some people that do not assign entries to multiple categories can still use the counting feature.

But people who assign multiple categories get no warning at all.
Instead the count is wrong and nobody seems to care.

Sorry, posting a wrong posting count is just plain wrong and this "feature" should be deactivated.
No, ultimately it needs to be fixed by a developer who knows a solution. Since Serendipity is OpenSource, everyone to whom this bug is important enough can contribute a fix.

Sorry, i tried that before posting here, but serendipity_fetchEntries() is not fixable, at least not for me. This function is controlled by a dozen parameters and another dozen global variables. How should someone like me - who just wants to fix a bug - know about the impacts of my change?
You can't expect someone fixing this function for you if this person isn't coding for S9y regularly.

By the way: just because Serendipity is Open Source this fact is no excuse for begging others to fix known bugs. It's ok for me if this feature is only available if postings have just one category. But returning/printing the wrong count number is just plain wrong behaviour and "only if it's important enough" is no excuse for bugs.

"Open Source" is not for adding bugs and dysfunctional features. This bad habit normally belongs to commercial companies.

Post by **garvinhicking** » Fri Sep 19, 2008 2:45 pm

Hi!

Instead the count is wrong and nobody seems to care.

There is a 3 or 4 years old bug in our bugtracker. Since nobody wants to fix this, I must only assume, nobody cares enough, that is true.

[qupte]Sorry, i tried that before posting here, but serendipity_fetchEntries() is not fixable, at least not for me. This function is controlled by a dozen parameters and another dozen global variables. [/quote]

Exactly. You just figured out why this fix is so complex and hard to do.

You can't expect someone fixing this function for you if this person isn't coding for S9y regularly.

Yeah, the one who does the fix will need to do some thorough testing and investigating.

By the way: just because Serendipity is Open Source this fact is no excuse for begging others to fix known bugs. It's ok for me if this feature is only available if postings have just one category. But returning/printing the wrong count number is just plain wrong behaviour and "only if it's important enough" is no excuse for bugs.

Expecting someone else to fix a bug for you with your motivating and kind words is no better.

This counting bug is personally totally irrelevant to me, I can live with that glitch just fine. Also other serendipity users have not cared more for the past 5 years.

When this feautre of counting was requested, it worked fine for the person who requested it, he was not using multiple category assignments, so that did the job for him and IMHO the feature is simply lacking another feature that makes it work for multiple categories.

I'm just saying it's a bug, and not a completely dysfunctional feature. Someone needs to fix it. I don't volunteer, and nobody can expect that from me. THAT is OpenSource. Everyone can code what he enjoys to, there is no reliability whatsoever. Read the license, and please stop complaining in your tone. Thank you.

Regards,
Garvin

judebert · Post by **judebert** » Fri Sep 19, 2008 3:20 pm

Open Source, ultimately, is about fixing and learning. Stallman couldn't fix a Xerox driver because they wouldn't let him, so he vowed to never use anything that he couldn't fix again. Later hackers posited a community of their brethren; these programmers could learn from each others' work, producing a steadily improving body of code, because by examining the source they could learn from each other.

It wasn't until later, especially with Linus, that we figured out the "with enough eyes, all bugs are shallow". We discovered that hackers like to scratch their own itches, and improve each others' code mostly when their itches overlap.

Serendipity fits perfectly into this framework. It's been Open Source from the beginning, one of the first FOSS blogs. It was made to scratch an itch: nobody had a fast, secure PHP blog. Other programmers found it and improved it to scratch their itches.

Non-programmers found it, too. They didn't learn from the code, but they did provide two useful services: testing and feature requests. We appreciate those contributions.

Garvin has spent a significant portion of his life improving, maintaining, and supporting s9y. He accepts others' code contributions, tests modifications, fixes bugs, and provides some of the friendliest and fastest forum support ever seen.

He's accomplished things that I had put in the "impossible" category. I don't think we can blame him for being unable to fix a single bug, especially when he's admitted the shortcoming and asked for help. Meanwhile, he's mitigated the bug, removing it from the default configuration.

If you've got insight into how this can be fixed, please share it. Otherwise, stop bothering the people who are getting work done. While we're always happy to respond to users, we get a little agitated when someone takes this kind of tone without even checking the bugtracker.

Post by **garvinhicking** » Fri Sep 19, 2008 3:40 pm

YFI: I just tried to look up the bug in the tracker but couldn't find it. But it was posted here in the forums before:

http://board.s9y.org/viewtopic.php?t=64 ... nt+archive

If we invented a car that could drive automatic on highways, but fails to do so on clumsy roads -- would we need to disable the working automatic highway-driving, just because the car goes off in rough territory? I believe not, the better way should be to improve the rough territory situation, but only with people who have knowledge in that territory and are interested to drive their care there.

That's my personal bottom line, as I've become way too agitated (like judebert expressed *g*) in this thread. I'm currently working 10-12 hours aside, WITHOUT the serendipity stuff that I want to occasionally check out here. OpenSource also lives from other people's contribution, I think a demanding tone to single developers is not the way to develop a product. You only get what you pay for, unless you do it yourself.

Ooops, again I wrote more than I wanted. I'll wait for tomorrow, I'll surely have a less agitated oppinion then

Regards,
Garvin

ads · Post by **ads** » Fri Sep 19, 2008 4:59 pm

Whoa, stop it!

judebert wrote:Open Source, ultimately, is about fixing and learning.

Yes, finding and fixing the problem was the first thing i tried, even before checking if someone else already posted about this one. In my initial post i wrote about this attempt and digged the problem done to one (big and heavily used) function.

I noted - and fixed - some PostgreSQL related bugs before. My posting counter is not very high so you can check my posting list if you want. But for this one i was unable to fix it or even could find the real source of the problem.

Garvin has spent a significant portion of his life improving, maintaining, and supporting s9y. He accepts others' code contributions, tests modifications, fixes bugs, and provides some of the friendliest and fastest forum support ever seen.

I appreciate this. I'm using Serendipity for my own blog and i recommend it to others and even help them to install and maintain the blog software.

Meanwhile, he's mitigated the bug, removing it from the default configuration.

If you will read my post again you will find that i don't have problem with not fixing this bug at all. But i find it impolite if this bug is already well-known and there's not even a warning in the plugin admin interface. A note like: "attention: if your posts have multiple categories, the posting count will be wrong" will tell everyone what's going on. This could be an easy and acceptable workaround for this problem - and this note could be in place since 2006:

garvinhicking wrote:YFI: I just tried to look up the bug in the tracker but couldn't find it. But it was posted here in the forums before:

http://board.s9y.org/viewtopic.php?t=64 ... nt+archive

Otherwise, stop bothering the people who are getting work done.

Nice attitude.
I was not the first and i will not be the last one who comes up with this bug. Sure, this one is not very important but just stating "it won't happen in the default configuration" is the kind of answer (and the kind of solution) i expect from companies like M$ who don't seem to care about the customer. Open Source is also about not hiding the problems in a dark corner and let repeat everyone the same steps to find out. Every possible change/bugfix/workaround for this bug would be a step toward a more user-friendly behavour.

While we're always happy to respond to users, we get a little agitated when someone takes this kind of tone without even checking the bugtracker.

You mean the Sourceforge bugtracker which is not linked from the S9y homepage? You are right, i did not check it - how should i know that S9y is using an external bugtracker without linking the website? Please add a prominent link on the homepage and next time i have a problem i will use this link before posting here.

But i checked the user forum and could not find a similar posting. The keywords in question always returned several hundred threads, even the keywords Garvins was using. Garvin proved me wrong and i want to excuse myself for opening another thread for the same problem.

Post by **garvinhicking** » Fri Sep 19, 2008 8:15 pm

Hi!

Try to replace inside your include/plugin_internal.inc.php file, class serendipity_archives_plugin, function generate_content at around this place:

Code: Select all

               $ec = serendipity_fetchEntries(
                    array($current_ts, $end_ts),
                    false,
                    '',
                    false,
                    false,
                    'timestamp DESC',
                    '',
                    false,
                    true,
                    'count(e.id) AS orderkey',
                    '',
                    'single',
                    false, $category_set // the joins used
                );

with this:

Code: Select all

               $ec = serendipity_fetchEntries(
                    array($current_ts, $end_ts),
                    false,
                    '',
                    false,
                    false,
                    'timestamp DESC',
                    '',
                    false,
                    true,
                    'count(distinct e.id) AS orderkey',
                    '',
                    'single',
                    false, $category_set // the joins used
                );

Adding a DISTINCT before e.id, and see if that might help.

Regards,
Garvin

ads · Post by **ads** » Fri Sep 19, 2008 9:10 pm

Code: Select all

                $ec = serendipity_fetchEntries(
                    array($current_ts, $end_ts),
                    false,
                    '',
                    false,
                    false,
                    'orderkey DESC',
                    '',
                    false,
                    true,
                    'count(DISTINCT(e.id)) AS orderkey',
                    '',
                    'single',
                    false, $category_set // the joins used
                );

This one works with PostgreSQL too.
Mind the changed "ORDER BY" field and i added () around the DISTINCT to make the code a bit more clear and portable.

Is it possible to skip the entire "ORDER BY"? This one is not necessary here and costs performance. In PostgreSQL it is required that you ORDER/GROUP the aggregate results.

Post by **garvinhicking** » Fri Sep 19, 2008 9:13 pm

Hi!

I believe skipping the order by statement by using "null" should work? I can't tell right now...

I believe we always used "DISTINCT field" instead of "DISTINCT(field)" in our other cases, because I believe the latter is not fully SQLite compatible...this would need to be tested?

Regards,
Garvin

ads · Post by **ads** » Fri Sep 19, 2008 9:22 pm

Oh, sweet, don't know about Sqlite ...
For PostgreSQL:

Code: Select all

ads=# select distinct(id) from test;
 id
----
  1
  2
  3
(3 rows)

ads=# select distinct id from test;
 id
----
  1
  2
  3
(3 rows)

Should work both ways ... i only prefer the clean way

Post by **garvinhicking** » Sat Sep 20, 2008 1:19 pm

Hi!

Okay, I'll add it as "distinct e.id" then. Did you check if the "order by" is essential or can be set to null?

Regards,
Garvin

ads · Post by **ads** » Sat Sep 20, 2008 1:34 pm

Both 'null' and 'false' lead to an empty "ORDER BY" clause in the query and thus to a syntax error in the logfile. If you can remove the ORDER BY at all that should be ok because without ORDER BY there's nothing to complain about for PostgreSQL.

Bug in Archive plugin

Bug in Archive plugin

Re: Bug in Archive plugin