Nope, not auto-generating HTML pages. It does send out emails to everyone who has opted to be notified of new links in specific categories and the global notification of any new link. Could that be causing the load?
Actually, server was upgraded with MySQL 4 in hopes of speeding up the site. However, now I'm getting an old problem, where it'll send multiple emails out to the same person. One person got 80+ emails when I added a new link and another got 100+.
Plus, the lag time is even higher. My website is unavailable for about 10 minutes after approving any link. Every other site on my server is fine - it's just the site that uses WSN Links that has this problem, so it'appears to be a script lag problem rather than server lag.
(EDIT: Just tested this myself and I've already received 70+ emails and they are still coming. Seems the entire time the site is lagged, it's blasting out emails).
Is there a simple way to shut off email notifications? I've been having too many problems with this since starting this site, so it's probably easier for me just to not use this feature at all. I think the script isn't able to handle this much traffic (and unfortunately, things are probably only going to get worse then).
(EDIT 2: I did a global replace and changed all notifylinks to "no" and the lag vanished when validating a link, and obviously no one is receiving duplicate emails because they are all set to "no")
This may also be some help. I happen to catch my site right when it was getting ready to crash. I checked my Apache status and I saw entries like this:
40-0 30176 0/548/13690 W 7.98 70 0 0.0 5.00 91.04 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 41-0 32165 0/0/9360 W 0.02 80 0 0.0 0.00 51.94 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 42-0 32166 0/2/11202 W 0.02 70 0 0.0 0.00 77.60 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 43-0 32169 0/0/10679 W 0.27 79 0 0.0 0.00 60.55 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 44-0 32170 0/0/8531 W 0.01 79 0 0.0 0.00 54.72 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 64-0 32198 0/1/1237 W 0.01 70 0 0.0 0.00 9.35 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 65-0 32199 0/2/1207 W 0.02 70 0 0.0 0.00 8.67 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 66-0 32200 0/0/2045 W 2.56 77 0 0.0 0.00 7.82 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 67-0 32202 0/1/957 W 0.01 77 0 0.0 0.000 12.47 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 68-0 32205 0/1/835 W 0.01 70 0 0.0 0.00 5.98 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 69-0 32206 0/0/292 W 0.55 76 0 0.0 0.00 0.81 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 70-0 32207 0/1/188 W 0.00 76 0 0.0 0.00 0.47 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 71-0 32209 0/0/292 W 0.81 76 0 0.0 0.00 0.95 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 72-0 32210 0/0/667 W 7.71 76 0 0.0 0.00 4.66 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 73-0 32211 0/0/59 W 0.01 76 0 0.0 0.00 0.55 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1 74-0 32212 0/1/39 W 0.01 76 0 0.0 0.000 0.21 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
There were plenty more, most looking just like the above processes. At first there were a few, then those processes kept multiplying over and over, bringing the site to a standstill, not completeing any of them and instead just adding more.
From my vantage point, it looks as if it gets stuck on a process, keeps trying to repeat it over and over, and then eventually crashes. Maybe that's kind of what's going on with the email - it keeps repeating it over and over again.
I've seen it do this before when it was near crashing, but never really paid attention to the entries (didn't notice they were all the same). This happens about once a day, where it crashes my Apache.
How about the above Apache logs? Do you know anything that would cause the script to hang on a process and repeat it over and over until Apache crashed?
This started happening as soon as I installed WSN Links on the server. I just didn't make the connection for a while, but my server used to be up for months at a time. After I installed WSN Links, it would crash Apache every few days, and over time more often as I got more traffic to the site.
I also am still getting duplicate emails about 50% of the time when someone reports a bad/broken link.
It's a shame, because I wanted to use WSN Links for a new site I started, but until I can figure out how to remedy my current problems, I can't risk another site causing my server to crash.
im having the same problem if you do find out what it is plz post here... if you have enough categories it is certain that it will crash your server
I expect that if you have many visitors you will not need many categories to make the server crash.. even without any categories or links the script takes about 1 to 2 seconds to load
My apache also just bogged down. Its done this a number of times, almost always between 9am-2pm during workdays, which is my peak user load. Today it looks like an Ask Jeeves spider tipped the scales for it.
I'm not autogenerating any pages.
I'm only sending e-mail on registration.
It was down for approx 45 minutes, so I restarted without investigating much
I did notice the below in my apache error log from various wsnlinks sites, but I think I've saw this before and don't think its a problem
PHP Notice: Undefined variable: upgradeinprogress in /var/www/html/mydir/start.php on line 7
I have a dedicated server, not sure about the specs.
My stats for the wsnlinks site in question is very similiar to synozeer, more unique visitors, but an equal number of page views (not sure about hits)
I'm not saying its WSNLinks, just that I have a smiliar story.
If it were a problem with the table not locking when sending emails, another visitor coming while the email is being sent and thus resending it would result in the email being fully filled -- because it's sending data pre-stored in the email table fully filled out, the person who's viewing the page at the time the queue sends it is unrelated to the person who the email is from.
And if it were an issue of the time when the email is actually created and stored in the email table, how in the world can one person's doing something in report.php be affected by someone else browsing the site?
Is the report comment problem reproducable on your copy of WSN Links? Where someone writes a comment, another person reports that comment and it says the person who reported it was the person who wrote it?
Other than that, I'm at a loss what to do. It's not even that big yet in terms of traffic, but I can't really promote my site because it's just going to crash, get slower, etc. I've already had to disable new link notifications and I'll probably have to disable new links in specific category notifications too because I'm getting duplicate emails there as well. I've tried everything on my server - upgraded php, mysql, tweaked my.cnf, diagnostics, paid server techs to check out my server, etc. The techs say it's the script that's causing the problems. So here I am.
Synozeer wrote: Is the report comment problem reproducable on your copy of WSN Links?
I used to have this problem regularly; so a while back, I removed the link to "Report Comment" from my "view comment" template. I also made it so that non-admins see a page that says "This feature is temporarily offline" if they do happed to find a link to the report comment page.
And here is the full text of the comment in question:
I see the e-mail logs in my admin panel. This site is running v3.10. I am planning on upgrading within the month to coincide with a redesign.
I've had the report comment problem using both qmail in the past and sendmail currently.
Also, the Ask Jeeves spider just tipped the scales, it was only hitting pages every 2-3 seconds, I just happend to notice that was the last entry in my logs when Apache died.
The crawl-delay is supported by ask jeeves/teoma, yahoo, and msn, however it isn't part of the defacto robots.txt standard and as such Google and other search engines don't pay attention to it.
I've also had server techs tell me that the script was using too many resources. As I make a living off my sites and couldn't easily change scripts I simply got a dedicated server. Now nobody complains about resources, but it also kills the server from time to time.
I used to have traffic growing exponentially and was spending several thousand a month on advertising. However the more people visited this site the slower the pages would load, and the fewer return visitors I would get (not to mention ad click through rates dropped during these slow peak hours) My traffic topped out several months ago at 2500 to 4000 uniques a day and hasn't grown much. Now I spend very little in advertising and just try to keep traffic steady.
To mitigate this I'm looking to upgrade the dedicated server to a 3.2GHz 2GB RAM, I'm also redesigning the site using no toplists, and manually updating static includes, category listings, etc. I will be looking switching off features one at a time and figuring out what will help the most.
I've looked at moving to mambo or similar systems, but found that the customization ability of WSN Links/WSN Forums is "LIGHT YEARS" ahead of any mambo/*nuke/blog software available.
If there is anything we can do to help resolve
1) Slow Response time under high loads 2) Duplicate e-mail
Then please let us know.
I personally get a complete ROI on my lifetime license of WNS Links every few hours. So if it is especially time intensive to concentrate on these issues then I'm sure that I (and possibly some others) would be more than happy to compensate you for your time.
If you can't see a starting point for this or can't take ownership because you can't reproduce it then let me know early so that I can start planning on a transfer to something else. As you know much better than the rest of us, it will be terribly expensive to customize any other script (let alone write from scratch) to reproduce the WSN Links functionality.
Its an absolutley wonderful script that is worth far more than you charge. It just doesn't scale well in the current form.
So I'm guessing none of the above problems will/can be resolved without a major rewrite?
I love WSN Links, but with my Apache crashing every 1-3 days and duplicate emails being sent out regularily, along with load time slower than it should be, it's not looking good for any future projects of mine (which I wish wasn't the case).
As I've said many times, none of the problems can be resolved unless I can see them. At http://links.webmastersite.net I make a test comment, report it, no problem. I got one and only one notification of the report in my email.
A rewrite would not help when I still wouldn't see how/where the problem happens. Somebody is simply going to have to figure out what unique thing they're doing and give me a process to reproduce it -- until then nothing can be done.
Of course when people let search engine spiders run wild unmetered on a dynamic site, if they get popular enough this will always kill their server and that is why it's the webmaster's responsibility to make an appropriate robots.txt. Spiders want to go through stuff as quickly as they can, they're happy to act as a DOS attack until unless you ask them not to.
0/5
1
2
3
4
5
This thread is closed, so you cannot post a reply.
Comments on WSN Links very resource intensive?
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
Nope, not auto-generating HTML pages. It does send out emails to everyone who has opted to be notified of new links in specific categories and the global notification of any new link. Could that be causing the load?
Adam
developer
Usergroup: Administrator
Joined: Dec 20, 2001
Location: Diamond Springs, California
Total Topics: 61
Total Comments: 7868
I suppose a sufficient number of notifications could.
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
Actually, server was upgraded with MySQL 4 in hopes of speeding up the site. However, now I'm getting an old problem, where it'll send multiple emails out to the same person. One person got 80+ emails when I added a new link and another got 100+.
Plus, the lag time is even higher. My website is unavailable for about 10 minutes after approving any link. Every other site on my server is fine - it's just the site that uses WSN Links that has this problem, so it'appears to be a script lag problem rather than server lag.
(EDIT: Just tested this myself and I've already received 70+ emails and they are still coming. Seems the entire time the site is lagged, it's blasting out emails).
Is there a simple way to shut off email notifications? I've been having too many problems with this since starting this site, so it's probably easier for me just to not use this feature at all. I think the script isn't able to handle this much traffic (and unfortunately, things are probably only going to get worse then).
(EDIT 2: I did a global replace and changed all notifylinks to "no" and the lag vanished when validating a link, and obviously no one is receiving duplicate emails because they are all set to "no")
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
This may also be some help. I happen to catch my site right when it was getting ready to crash. I checked my Apache status and I saw entries like this:
40-0 30176 0/548/13690 W 7.98 70 0 0.0 5.00 91.04 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
41-0 32165 0/0/9360 W 0.02 80 0 0.0 0.00 51.94 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
42-0 32166 0/2/11202 W 0.02 70 0 0.0 0.00 77.60 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
43-0 32169 0/0/10679 W 0.27 79 0 0.0 0.00 60.55 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
44-0 32170 0/0/8531 W 0.01 79 0 0.0 0.00 54.72 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
64-0 32198 0/1/1237 W 0.01 70 0 0.0 0.00 9.35 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
65-0 32199 0/2/1207 W 0.02 70 0 0.0 0.00 8.67 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
66-0 32200 0/0/2045 W 2.56 77 0 0.0 0.00 7.82 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
67-0 32202 0/1/957 W 0.01 77 0 0.0 0.000 12.47 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
68-0 32205 0/1/835 W 0.01 70 0 0.0 0.00 5.98 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
69-0 32206 0/0/292 W 0.55 76 0 0.0 0.00 0.81 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
70-0 32207 0/1/188 W 0.00 76 0 0.0 0.00 0.47 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
71-0 32209 0/0/292 W 0.81 76 0 0.0 0.00 0.95 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
72-0 32210 0/0/667 W 7.71 76 0 0.0 0.00 4.66 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
73-0 32211 0/0/59 W 0.01 76 0 0.0 0.00 0.55 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
74-0 32212 0/1/39 W 0.01 76 0 0.0 0.000 0.21 24.232.221.107 gamelinks.com GET /link.php?action=detail&id=1246 HTTP/1.1
There were plenty more, most looking just like the above processes. At first there were a few, then those processes kept multiplying over and over, bringing the site to a standstill, not completeing any of them and instead just adding more.
From my vantage point, it looks as if it gets stuck on a process, keeps trying to repeat it over and over, and then eventually crashes. Maybe that's kind of what's going on with the email - it keeps repeating it over and over again.
I've seen it do this before when it was near crashing, but never really paid attention to the entries (didn't notice they were all the same). This happens about once a day, where it crashes my Apache.
Does this give any clues as to what's going on?
Adam
developer
Usergroup: Administrator
Joined: Dec 20, 2001
Location: Diamond Springs, California
Total Topics: 61
Total Comments: 7868
Is there a simple way to shut off email notifications?
Removing the admin email does it. Email prefs control more specific things. Anyhow, I've seen no way to reproduce duplicate email issues.
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
How about the above Apache logs? Do you know anything that would cause the script to hang on a process and repeat it over and over until Apache crashed?
This started happening as soon as I installed WSN Links on the server. I just didn't make the connection for a while, but my server used to be up for months at a time. After I installed WSN Links, it would crash Apache every few days, and over time more often as I got more traffic to the site.
I also am still getting duplicate emails about 50% of the time when someone reports a bad/broken link.
It's a shame, because I wanted to use WSN Links for a new site I started, but until I can figure out how to remedy my current problems, I can't risk another site causing my server to crash.
Adam
Member
Usergroup: Customer
Joined: May 24, 2005
Total Topics: 1
Total Comments: 16
im having the same problem if you do find out what it is plz post here... if you have enough categories it is certain that it will crash your server
I expect that if you have many visitors you will not need many categories to make the server crash.. even without any categories or links the script takes about 1 to 2 seconds to load
Forum Regular
Usergroup: Customer
Joined: Feb 19, 2004
Location: Michigan
Total Topics: 57
Total Comments: 185
My apache also just bogged down. Its done this a number of times, almost always between 9am-2pm during workdays, which is my peak user load. Today it looks like an Ask Jeeves spider tipped the scales for it.
I'm not autogenerating any pages.
I'm only sending e-mail on registration.
It was down for approx 45 minutes, so I restarted without investigating much
I did notice the below in my apache error log from various wsnlinks sites, but I think I've saw this before and don't think its a problem
I have a dedicated server, not sure about the specs.
My stats for the wsnlinks site in question is very similiar to synozeer, more unique visitors, but an equal number of page views (not sure about hits)
I'm not saying its WSNLinks, just that I have a smiliar story.
developer
Usergroup: Administrator
Joined: Dec 20, 2001
Location: Diamond Springs, California
Total Topics: 61
Total Comments: 7868
On the undefined variable, please read the manual. scripts.webmastersite.net/w...nks/wsnmanual/articles/238
Today it looks like an Ask Jeeves spider tipped the scales for it.
Well, do you have a crawl-delay set in your robots.txt? 60 is a good value. If not, that's likely to happen.
On duplicate emails, again, I can't do anything until you can show me how to reproduce it.
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
On duplicate emails, again, I can't do anything until you can show me how to reproduce it.
I posted some more info in this thread:
https://www.webmastersite.net/forums/thread/5008
Was that useful at all?
-Adam
developer
Usergroup: Administrator
Joined: Dec 20, 2001
Location: Diamond Springs, California
Total Topics: 61
Total Comments: 7868
I'm not sure.
If it were a problem with the table not locking when sending emails, another visitor coming while the email is being sent and thus resending it would result in the email being fully filled -- because it's sending data pre-stored in the email table fully filled out, the person who's viewing the page at the time the queue sends it is unrelated to the person who the email is from.
And if it were an issue of the time when the email is actually created and stored in the email table, how in the world can one person's doing something in report.php be affected by someone else browsing the site?
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
Is the report comment problem reproducable on your copy of WSN Links? Where someone writes a comment, another person reports that comment and it says the person who reported it was the person who wrote it?
Other than that, I'm at a loss what to do. It's not even that big yet in terms of traffic, but I can't really promote my site because it's just going to crash, get slower, etc. I've already had to disable new link notifications and I'll probably have to disable new links in specific category notifications too because I'm getting duplicate emails there as well. I've tried everything on my server - upgraded php, mysql, tweaked my.cnf, diagnostics, paid server techs to check out my server, etc. The techs say it's the script that's causing the problems. So here I am.
Adam
Forum Regular
Usergroup: Customer
Joined: Feb 19, 2004
Location: Michigan
Total Topics: 57
Total Comments: 185
Is the report comment problem reproducable on your copy of WSN Links?
I used to have this problem regularly; so a while back, I removed the link to "Report Comment" from my "view comment" template. I also made it so that non-admins see a page that says "This feature is temporarily offline" if they do happed to find a link to the report comment page.
Example http://www.securitydocs.com/reportcomment.php?id=3333
Yet this morning I had about 600 e-mails that said the below:
I see the e-mail logs in my admin panel. This site is running v3.10. I am planning on upgrading within the month to coincide with a redesign.
I've had the report comment problem using both qmail in the past and sendmail currently.
Also, the Ask Jeeves spider just tipped the scales, it was only hitting pages every 2-3 seconds, I just happend to notice that was the last entry in my logs when Apache died.
The crawl-delay is supported by ask jeeves/teoma, yahoo, and msn, however it isn't part of the defacto robots.txt standard and as such Google and other search engines don't pay attention to it.
I've also had server techs tell me that the script was using too many resources. As I make a living off my sites and couldn't easily change scripts I simply got a dedicated server. Now nobody complains about resources, but it also kills the server from time to time.
I used to have traffic growing exponentially and was spending several thousand a month on advertising. However the more people visited this site the slower the pages would load, and the fewer return visitors I would get (not to mention ad click through rates dropped during these slow peak hours) My traffic topped out several months ago at 2500 to 4000 uniques a day and hasn't grown much. Now I spend very little in advertising and just try to keep traffic steady.
To mitigate this I'm looking to upgrade the dedicated server to a 3.2GHz 2GB RAM, I'm also redesigning the site using no toplists, and manually updating static includes, category listings, etc. I will be looking switching off features one at a time and figuring out what will help the most.
I've looked at moving to mambo or similar systems, but found that the customization ability of WSN Links/WSN Forums is "LIGHT YEARS" ahead of any mambo/*nuke/blog software available.
If there is anything we can do to help resolve
1) Slow Response time under high loads
2) Duplicate e-mail
Then please let us know.
I personally get a complete ROI on my lifetime license of WNS Links every few hours. So if it is especially time intensive to concentrate on these issues then I'm sure that I (and possibly some others) would be more than happy to compensate you for your time.
If you can't see a starting point for this or can't take ownership because you can't reproduce it then let me know early so that I can start planning on a transfer to something else. As you know much better than the rest of us, it will be terribly expensive to customize any other script (let alone write from scratch) to reproduce the WSN Links functionality.
Its an absolutley wonderful script that is worth far more than you charge. It just doesn't scale well in the current form.
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
So I'm guessing none of the above problems will/can be resolved without a major rewrite?
I love WSN Links, but with my Apache crashing every 1-3 days and duplicate emails being sent out regularily, along with load time slower than it should be, it's not looking good for any future projects of mine (which I wish wasn't the case).
Adam
developer
Usergroup: Administrator
Joined: Dec 20, 2001
Location: Diamond Springs, California
Total Topics: 61
Total Comments: 7868
As I've said many times, none of the problems can be resolved unless I can see them. At http://links.webmastersite.net I make a test comment, report it, no problem. I got one and only one notification of the report in my email.
A rewrite would not help when I still wouldn't see how/where the problem happens. Somebody is simply going to have to figure out what unique thing they're doing and give me a process to reproduce it -- until then nothing can be done.
Of course when people let search engine spiders run wild unmetered on a dynamic site, if they get popular enough this will always kill their server and that is why it's the webmaster's responsibility to make an appropriate robots.txt. Spiders want to go through stuff as quickly as they can, they're happy to act as a DOS attack until unless you ask them not to.