Printer-friendly version
Forums:
Taxonomy upgrade extras:
Did anything change recently with the Topshelf software? I've been using firefox browser, with the down-them-all add-on to batch-download the new posts daily, so I could read them on my phone. The site was down on the 2nd March when I visited, and then when it came back, down-them-all no longer works. Does anyone else read the same way? Have you hit the same problem?
What changed...
We rebuilt part of our database. Since nothing in our database can really affect the functions of an add in to your browser, I don't see exactly how this can be an effect caused from something we did.
But... Several other people have complained about some sorts of problems after that rebuild or from just before it which was why we did the rebuild. Your problem does not seem related to our database problem but who knows since I don't actually understand what you're problem is. Saying somehting doesn't work is not actually that informative. :)
What doesn't work? How did it used to work? What did you do to make it work? Honest, I would try to help you but I have no clue to how to work on your problem.
At least two other people are having similar browser problems after the rebuild with functions in their browsers that do not seem to be related to anything on our end. But... the timing is curious to say the least.
More info on problems of this sort is needed. I'm not even sure what questions to ask except that I need more details.
Hugs,
Erin
= Give everyone the benefit of the doubt because certainty is a fragile thing that can be shattered by one overlooked fact.
I also have an issue. I use
I also have an issue. I use Instapaper to download stories for reading later. Sometime Friday I started getting a "Not available for download" message. I'm only having that problem here.
However, it started happening after the database rebuild.
If I can offer more details, please ask and I'll offer what help I can.
Details on how?
Instapaper is a module but we don't have it installed here. Is that an add on to your browser? What browser? What exactly do you do and on what stories have you got this error?
Not available for download does not sound like a message being given by BC or Drupal so I'm not sure what is happening. Can you clear your local caches? I'll try rebuilding permissions for your account, just in case they got mangled in the rebuild but I'm not sure what if any effect that will have.
Hugs,
Erin
= Give everyone the benefit of the doubt because certainty is a fragile thing that can be shattered by one overlooked fact.
Erin, thanks for your
Erin, thanks for your consideration. Sorry about double posting.
Instapaper (The app) runs on my tablet as a downloader and a reader.
I use Chrome as my browser. When I encounter a website I want to read later, I go to share then click on Instapaper. At that point the URL gets saved to Instapaper's website. I also have a "read later" bookmarklet that I use with Firefox and SeaMonkey depending on which computer I'm using. Then Instapaper downloads the page, reformats it for my tablet and makes the reformatted site available. The Instapaper app then downloads it so I can read it later.
The "not available for download" message is coming from Instapaper. I think that it means (I couldn't find any documentation on the error message) that when Instapaper tries to download the page to reformat it, it gets an error. This most commonly happens to me when an article is behind a pay wall.
My answer is kind of long winded, I hope it answers what you need to know.
Go to share?
Not sure what Go to Share means? Is share a menu in your version of Chrome? Sounds like the Instapaper website is being blocked from downloading. Hmm. Instapaper's website says that site's that block crawlers will block downloads. We do block some crawlers, mostly ones from Asia and Eastern Europe.
I wonder if FDC the server farm has put something up in their firewall due to the DDOS attacks they've been having?
Gives me something else to investigate.
Hugs,
Erin
= Give everyone the benefit of the doubt because certainty is a fragile thing that can be shattered by one overlooked fact.
Thanks for the quick reply
Thanks for the quick reply Erin.
I'll give what info I can.
down-them-all has a selection screen where it automatically selects files by scanning for "/bigcloster.us/topshelf/fiction" which marks 99 files when I browse to page one of "http://bigclosetr.us/topshelf/view/3months"
It would download the file and rename them to name.ext in a folder I've selected.
For instance
http://bigclosetr.us/topshelf/fiction/43025/id-part-2
down-them-all logs aren't too helpful, but there are javascript reports of "unknown filename" so I suspect the filename format may have changed?
I don't know if this matters, but when I browse to a chapter, then try to 'save as', the default filename comes up differently to how I remember it. But given I don't usually read this way, please don't give this too much weight.
for example "id (Part 2).html" comes out as "id (Part 2) | Bigcloset TopShelf.html"
Brackets have sometimes caused problems, but the pipe symbol (|) might do as well. And the ' Bigcloset Topshelf' seems new...
I hope that's some help.
None of that is very helpful ...sigh...
'Save as' is a function in your browser, when you activate it, it doesn't send any messages to our software but works with whatever it has in memory. Filename formats have not changed.
You say above "It would download the file...". What file? Each file of 99? One big file? What would be in such a file?
None of this seems all that relevant to what I know of how our site functions.
Piper, Cat, Sephrena and I will keep plugging away on this problem but if it is really on our end, I'm at a loss as to what to even look at.
Hugs,
Erin
= Give everyone the benefit of the doubt because certainty is a fragile thing that can be shattered by one overlooked fact.
Direct downloading fails
I just tried downloading a single story from the http://bigclosetr.us/topshelf/view/3months page using DownThemAll and got a "temporary error" message.
After that I tried the same directly using wget:
wget -nd http://bigclosetr.us/topshelf/fiction/43031/altered-fates-boss
--2013-03-03 20:14:23-- http://bigclosetr.us/topshelf/fiction/43031/altered-fates-boss
Resolving bigclosetr.us (bigclosetr.us)... 204.45.253.66
Connecting to bigclosetr.us (bigclosetr.us)|204.45.253.66|:80... connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2013-03-03 20:14:25 ERROR 500: Internal Server Error.
Martina
Wget is blocked
Has been blocked for ages. People in eastern Europe were using Wget to download the ENTIRE SITE AT ONE TIME! Sheesh!
Sorry but we can't allow such a huge hole to be exploited.
Hugs,
Erin
= Give everyone the benefit of the doubt because certainty is a fragile thing that can be shattered by one overlooked fact.
DownThemAll fails to download story page.
I did a quick trace while using DownThemAll do download a single story.
DTA uses a Firefox user-agent but maybe is still detected and blocked.
The whole page with the story gets sent but with a "HTTP/1.1 500 Internal Server Error" and it looks like DTA doesn't save the page to disk because of that error.
Martina
It actually does still work,
It actually does still work, but you have to truly know how to use it. I use it on several sites when I want to download a single story. I only use it to do an entire site when I'm trying to set up a local mirror - this site has too many links to try doing that. It's stupid to try doing a mirror of a database driven/dynamic content site _anyway_. You'd be better off offering some money to just ask for a copy of the damned database for a local server. (I've thought about it!)
As to how you get around the blocks? I'm not going to say. Period. If Erin or Piper asks, I'll let them know, but that's it.
I'll get a life when it's proven and substantiated to be better than what I'm currently experiencing.
It would potentially download
It would potentially download the 99 files, except it's set to skip already downloaded files so would normally only get that days new episodes etc.
I suspect that what will have
I suspect that what will have to happen is to purge this site from those particular plugins, or remove the plugins completely, purge the cache, and reinstall the plugin.
I'll get a life when it's proven and substantiated to be better than what I'm currently experiencing.
Sounds like an idea
At times i have got the damnedest behavior on tried and trusted websites, closed browsers, logged out of my computer, logged back in and restarted browser and everything was fine.
I'm not saying that that kind of thing would solve any of these problems but I'm at a loss as to how anything on our end could be producing some of them.
Hugs,
Erin
= Give everyone the benefit of the doubt because certainty is a fragile thing that can be shattered by one overlooked fact.
something changed?
Yes, I have been e-texting about this problem with the people here at Big Closet for the past day +.
They gave me a list of recommendations, none of which helped but who knows, in your case they might.
They also brought up some issues which neither I nor our company need so we are going to have some pros come in and look our stuff over just to be on the safe side.
It still smells greatly of rotten sardines that this all happened when the site went down twice for short periods after the major downtime for the "rebuild". I don't believe the "rebuild" is at fault because after it was "completed" I could still go to the site, not bother to log on, and download story chapers of either my own or any other which was on the front page.
I vaguely remember something was mentioned about making some changes due to hackers in either a post or a message after the rebuild. perhaps something done then is also afflicting (I chose that word intentionally) our downloading.
Anesidora
The change in regard to hackers has been undone
But at any rate, the hacker type stuff would have given you messages when you tried to post, not when you tried to download. As far as I know, none of our software is even looking for a right click on anything, if I'm understanding what your problem is.
I really need more information about what you are doing and what the symptoms are. I have only the vaguest idea of just what is going on at your end.
Hugs,
Erin
= Give everyone the benefit of the doubt because certainty is a fragile thing that can be shattered by one overlooked fact.
I just re-tried down-them-all
I just re-tried down-them-all after reading you'd disabled an anti-hacker feature. It's not made any difference to the problem I'm seeing unfortunately.
If you bring in people to
If you bring in people to check things over at your business, feel free to contact me to bounce their suggestions off of. That's one of the things I do - try to keep *mumble* numbers of sites safe/clean. I can guarantee that I wouldn't be able to do anything _for_ you - it has to start locally, and I'm not - but I can help with suggestions/experience.
(No charge)
I'll get a life when it's proven and substantiated to be better than what I'm currently experiencing.
Maybe you could just make
Maybe you could just make screenshots from what happens and what should happen.
I can still save site pages not logged in
from any of our sites. even on a neighbors pc, NOT LOGGED IN. As for the DownThemAll, trying to stream more than one download file at a time from any site runs the risk of either the site owner (Which in this case I know is not her doing this) or the farm/hub owner to see this, and consider it an attack or bandwidth hogging, and block the ability to do multiple download requests at the farm/hub level.
Can be, not that they always do that.
All of you have to understand 1 thing, we do have to change how we operate for obvious security reasons - we are being pounded daily and so is our server farm we belong to. If you want us up and to be able to visit us and do what you do, we have to tighten up against the attacks. Blame the DDoS attackers, not us for having to defend against them.
So please do not wig out or accuse us because of these changes. Programs update, things get blocked because of security loopholes and such.
We will try to help you resolve the issue.
Sephrena
by default down-them-all
by default down-them-all would download four files at a time, but that had been mentioned as a possibility in one of the Google results I found while searching for solutions, so I'd set it to only download one-at-a-time.
> We will try to help you resolve the issue.
Thanks for that, I appreciate it. You'd be quite within your rights to tell us to go away as the basic functions of the site all seem ok.
Y'know, I really don't
Y'know, I really don't understand the people trying to attack the site. It's almost as if there's something _other_ than the normal reasons for attack involved.
I'll get a life when it's proven and substantiated to be better than what I'm currently experiencing.
Interesting...
Interesting. While on the topic of Firefox add-ons that have suddenly stopped working on this site after the recent outage (but worked fine before the outage, and still works fine on other sites), perhaps I should mention another add-on...
I use one called "Interclue", which when I hover my mouse pointer over any hyperlink causes a pop-up window (known as a "Clueviewer") to appear that let's me look at whatever is on the other end of that link, without actually clicking on it. Useful, especially when I see a comment posted that has an interesting title, but I am not sure if I want to actually open the story up to read everything in it. [I am not certain, but I suspect that Interclue also looks *exclusively* at the comment, without making Drupal re-generate the entire story or blog page... which I *hope* (but do not know for certain) means that I am not running up the story "Reads" counter just by peeking at a comment, when I have not actually read the story. Interclue says that using Clueviewer on a link is the same as clicking on it as far as the website is concerned, but that doesn't seem to be entirely true -- since I used to only see the one comment that the link pointed at, not the entire story and all the comments there...]
Shrug. Anyway, the way it used to work is that the Clueviewer window would pop up showing the comment (not usually the whole story or blog that the comment was attached to... although that does sometimes happen for very recent comments for some reason, in which case I would have to actually click on the link to read the "brand new" comment"). The Clueviewer window stays there so long as my mouse is either over the link, or moved into the Clueviewer window itself, but automatically fades away if I move the cursor elsewhere. Unfortunately, what is happening now for every single internal link on the BCTS pages (although not the external links to other sites, which still work fine... except a couple that I think are also actually on the same servers as BCTS...), is the following error message, slightly edited to just show the relevant text and skip the graphics and hypertext links.
The following is what I get when I skip back out to the main page and use Clueviewer to look at a comment in this thread. It happens regardless of whether I am logged in or not... and I repeat, it did *not* happen here prior to the site crash...:
Shrug. I can live without Interclue working right here, but it is sort of annoying. I just mention it as it seems to be a symptom of a larger problem with a bunch of software and this site, which all seems to have happened about the same time... so I thought you might like to know about it, too. By the way, as the error message refers to "Default Timeout" settings -- that is not the problem here. The error message pops up immediately, not after ten seconds.
Smile. But as I said... this is just for your information in possibly tracking down something odd in the software. I am not complaining about it.
Testing needed!!
Can people who have been having problems make a few new tests?
We actually found a problem in the access file. This may fix the problem.
Hugs,
Erin
= Give everyone the benefit of the doubt because certainty is a fragile thing that can be shattered by one overlooked fact.
Sorry...
Not sure if I was one of those people you wanted to run a test... but Interclue still does not work on this site -- "Error 500" again. [But it does work on another site I have open in another tab right now... just as previously].
It's fixed now!
at least as far as down-them-all in Firefox is concerned. Thanks very much for all your work :)
Funny Little Thing....
Before the outage, any story I read on the main menu would change color-link on page reload. Now it doesn't... mostly?
I've had just to of many that still changed to reflect being read. Strange little thing...
Ideas from the peanut gallery
Oh. Now is a good time to say that I have long been impressed with the very intelligent and professional way BCTS is run. I have meant to compliment you for the longest time. Sorry that it took the occasion of this trouble to finally bring it out.
Daddy always said that free advice is usually worth about what you pay for it, but here goes.
The database rebuild probably included re-installing the Web server itself(?) A re-install that might have affected in some small way either what ports are blocked and/or name server caches(??) Either because a re-install unintentionally took some config setting back to default or because the mere reinit dropped in some config change that had been made in the last week or whatever but re-init didn't take place at that time, but only took place this Sunday evening(???)
If anything has changed in the way the BCTS's host is resolving the name, might that cause most of this? Meaning that DNS caches are still reflecting a now out of date address, perhaps. Maybe your host is using something like the old 'rot' daemon to distribute the load and the ip addresses are now shared out slightly differently, or some similar slight change took place there?
Um, this is probably dumb but should we be asking if this is at all connected to something being not quite up to date with IPv6 at your site? Something that would affect only a very few users and leave most of us just fine?
I have been out of the tech trenches for years (plus I quite literally have brain damage) so thank you if you were polite enough to read this all the way through, much less take any of the thoughts seriously. :-)
Supportively,
Annie
500 Error (using Chrome but can still browse)
I have noticed over the last few days that the BCTS Server is returning an http status of "500 Internal Server Error" rather than the normal "200 OK". We can still browse the site as the page is returned regardless, however some web tools will see the 500 response and die.
This can be seen by using Chrome's Tools > Developer Tools > Network then refresh the page. (Firefox/Firebug has similar display. Great for finding references to missing files etc)
Based on the few M$ type sites I help with it usually implies that the server has had a problem in making up the page and there is probably a log file somewhere filling up with messages. Sorry I don't know enough about Drupal to guess at anything server-side, but I can guess that the 500 response will be causing havoc with some RSS readers and other tools.
Hope that helps a little,
Cheers, Kiwi.
Good catch. I just did a test
Good catch. I just did a test with Chrome as you suggested and got the 500 status followed by the page contents.
I had a problem with "Readability", a Chrome extension, not working. I reported it to them and it now works, so perhaps they modified their software to ignore the unhelpful 500 status.
Kris
{I leave a trail of Kudos as I browse the site. Be careful where you step!}