/dev/random

I have rarely seen programming taking advantage of Referential Integrity constraint e.g. CASCADE on UPDATE. It's immensely useful specially in large application where data is subject to change by multiple people. Here is another good example of referential integrity in MySQL http://javarevisited.blogspot.com.ar/2012/12/what-is-referential-integrity-in-database-sql-mysql-example-tutorial.html which explains cascading.

Just an update...We applied the robots.txt fix and noticed a distinct drop in origin traffic and CPU usage.

Not all magpie-crawler connections have gone but most have. We will only block the IPs as the last resort but it may come to that if we can't get rid of them all via robots.txt.

Seems magpie is (as stated above) simply overly aggressive.

Hi Eli,

Unfortunately I ended up having to make the decision, as I knew I would. One thing you learn when owning pets is that you will outlive them, and that more often than not you are going to have to make some hard decisions. What you have to remember is that you have to act in the best interests of your pet. It is not betrayal to end suffering, although the point at which you decide that quality of life has gone will be different for different people and different situations. In fact, I see it as a duty, a responsibility of ownership.

I hope that your situation is easy on both of you, but please, don't shy away from the hard decision if and when it becomes necessary. You will have doubts, that is only human - but if you love your dog the way it comes across from your words you will know when it is time to act.

I took my twelve yr old golden retriever for a checkup and although she had been put on a reduced calorie diet and lost some weight and her arthritis was starting she had some really good days. The shock was the blood test showed she has leukemia and will die in 30 to 60 days. She shows no signs except the blood counts, eats well, no stomach problems but has anemia which slows her down. I am hoping I will not have to make that choice unless she stops eating or drinking as I feel I would feel I am betraying the animal that loves me most and will do anythings to please me. I understand how you feel pray so that he goes before you have to make a choice and he does not suffer.

Hi all

The magpie crawler is also causing us some nasty issues on several large sites. The main issue for us is in fact that they persist in crawling non-existent pages, one of which alone is causing us over 20,000 404s per day.

We are taking steps now to ban magpie from our servers.

Cheers
Neil

You are mistaken on a few fronts here. Firstly there is no error in the syntax, and the syntax used is 'best practice' for MySQL. Secondly, there is no single quote, they are backticks. Lastly it is not recommended to leave these out, as you set yourself up for problems in the future.

As to captcha - I'll look into that.

Note - the syntax that you used is wrong

You have mentioned : CREATE DATABASE "bookshop';

Firstly - one can use either small oor caps for the create database

Secondly - if you give a single quote it will throw a ERROR 1064

Therefore, the correct syntax is as follows

create database sonic;
(no quotes)

Best....
(also one comment on the website caoptcha code - there is some problem, it refreshes the some captcha multiple times and does not allow to post, until one reloads the page and gets a new captcha. Also your captcha is little strange characters and one cant figure out the letters)

They're being a nuisance for me as well. Whole subnet blocked now, for all servers.

Here are some more:
Don't indent your code, or better yet, indent it randomly.
Put inexplicable white space in here and there to make the file longer.
Always leave out of date 'to-do' comments in place, and never change a comment when you change the code.
If you must comment your fixes, be sure to leave out any meaningful references like CR or QC numbers, eg "// change (jc)"
Finally - and it's not really a coding issue, but it does make for a good laugh - tell your boss that everything's fixed and he can tell his client to relax, then check in the wrong version of the file!

Thanks for the writeup!

Galera Replication is missing in your reference list. Galera provides synchronous multi-master clustering for MySQL/InnoDB. And it is true multi-master, you can write to any row at any cluster node - just like MySQL/Cluster but works on InnoDB. And fully open source, downloads here: http://www.codership.com/downloads/download-mysqlgalera/

Thanks Giuseppe, I should have remembered that. If I've forgotten anyone else, please chime in with a comment.

Multiple masters replication, without the limitations found in MySQL, can be done easily using Tungsten Replicator (see http://code.google.com/p/tungsten-replicator/wiki/TRCMultiMasterInstallation#Multi-Master_Installation).

Hi Henrik,

From my personal perspective I'd say that it isn't a matter of sites being proactively shut down, but a "death of a thousand cuts". Ignoring the community sites is easy, they don't provide any direct bottom-line benefit, they require resources that could be deployed in efforts that are more aligned to revenue generation, and they allow a platform for uncontrolled commentary. A company that is so tightly centralised as Oracle doesn't have that sort of platform in their DNA. Unlike MySQL AB, which understood that the community would become their customer base, I'm not sure Oracle sees the benefit of Community. Users, yes, but users in Oracle's thinking are paying customers.

We are already seeing signs of the lack of commitment to Community. Take a look at the forums, or MySQL Forge and tell me if you see signs of active engagement and a commitment to the community.

As to who should build a replacement and what it would be called, there are issues there. The name "MySQL" is a registered trademark, and if the site attracted a huge following you can bet the Oracle lawyers would take a dim view of any use of the trademark. However there are precedents and many products have community-run support sites that are not affiliated with the company producing the product. It could be done. But who should do it? It probably doesn't matter much. As long as the future was assured and the site was unlikely to be dropped due to pressure from parties with their own agenda, who actually provides the platform is almost irrelevant.

The main thing required for any Community site is active moderation and management. I engaged with the community to put in place moderators for the MySQL forums, rather than have all of the moderation done by the ever-shrinking internal resources (the Community team was effectively decimated long before I left, and as mentioned, the web team was not doing much better). What I couldn't do was give them the resources they needed to effectively take control of the site. I think that any new site should have community engagement and have a means by which the community can effectively manage the site. There are limits, of course, but making the site as much a community resource as possible while protecting it from misuse is a worthwhile although delicate balancing act.

Thanks for sharing!

Suppose some community site is actually shut down in the future... In your opinion, what should we do then? Who should build a replacement site and what would it be called?

Francis, it is nearly impossible to answer the question without knowing more about your setup, and this probably isn't the right forum. Millions of records should not be an issue, there are plenty of sites out there with many millions of records in MySQL databases not having issues with single table queries.

You really should first check that you are using indexes effectively. This is pre-empting a future episode, however you can use the EXPLAIN syntax to find out how the query is likely to use indexes. The episode "Creating your first database" does try to explain the use of indexes and why you need them. You might want to take another look at that.

If you want more information, there are plenty of places where you can get some interactive help. There is the #mysql channel on irc.freenode.net, and the official forums site at http://forums.mysql.com/.

Hello i have this millions of records and im having problems with the retrieval of the data because its very slow. even for just fetching a single table what is the problem and what is the solution? any idea thanks in advance

Thanks, Sheeri, a very good point. Hopefully I explained both the correct way and why the short version was used in that particular example, however you are correct, people will often just use the example code.

I recommend naming fields in your INSERT. Not doing so saves keystrokes but at the expense of having your code break if you add a field.

Remember that folks will copy and paste your example, so you have to set a good one!

Wowzers, reading your comments about this book definitely wants to make me get it. I must admit that I'm not into SF books BUT I did read "Ender's Game" a few years ago and I LOVED IT! It absolutely blew my mind. I had to read it for school and thank God I did or else I would have never stumbled upon it. I even told my younger brother to read it and he did. He also loved it. Either way. I might just have to trust you and get "Overlocked"

And I can't believe you got it signed! And haha you read it in less than 24 hours. Wow, trooper!

-Taylor

I also had to block access via .htaccess

Thanks for the tip, their "crawler" still misbehaves.

Great rant! I've run across similar code and want to add one more way to write crappy code: Use meaningless file names!

I've had to work on code bases with names like leftFish.inc.php (And not on code that had anything to do with aquariums either :( )

Welcome Adam!

Adam,

A very warm welcome!
I am sure you will enjoy working at SkySQL!

Let's make our first days a great and enjoyable experience to share with all!

-ivan

Welcome Adam!

The exact same problem. Their entire range 94.28.34.192/26 has been blocked from all of my servers. Leeches!

I've managed to cut off some of the flow into the chook yard and get some drainage out the other side so it's still a part bog / part lake - but not quite as bad as it was.

But the wind has come up now and they are talking possible destructive - but you wouldn't bloody credit it - I'm up to my ankles in water and the bloody sun's just come out!

Weather bureau is talking possible thunderstorms - wouldn't be surprised it's 15 degrees outside and with that sun out - sheesh.

That big redgum in the soak - is now in the middle of a large fast running stream! I've got some photos - should have taken the video camera though - might try and get down there after a shower and a coffee.

And these issues are tripled if you have to mess with the DNS for them!

Some of the developers of the MySQL Enterprise Monitor have been running 5.1 for a year or more. For us this had more to do with validating that we were ready than validating the mysqld -- for our uses, the 5.1 mysqld been solid for a long time. Me? I switch around, and even play with 6.x sometimes; speaking of, I need to give 6.0.8-alpha a whirl ....

What does Philip K. Dick wishes to say about the american society in 1968? can anyone tell me? I am doing a research of the book..

It it great that this won't require the proxy because that would limit its use.

I can't restart apps to use a different port to talk to the server. I expect many of us have that constraint.

I wouldn't put the proxy in front of a critical server. That is just one more point of failure. And I have yet to see dbt2 or sysbench results that include the use of deployment. Is the proxy still single-threaded? Does it run Lua on every query and result?

Just on getting data from Proxy, have you seen the interview with Mark on the DevZone?

Take a look down towards the bottom at the answer to the question "Are you exploring ways to use MySQL Query Analyzer without needing Proxy middleware?" where Mark states:

"We don’t want MySQL Proxy to be the only source of the metrics we are collecting. How you get this data into the Service Manager is pretty flexible and is based on REST. There are some other sources we want to explore. We would like to consume log data, we would like to consume the slow query log, or the general query log, even if it is sampled from time to time, so that you don’t have to use MySQL Proxy to get this information.

Also, there are ways to instrument the MySQL Connectors to get this information, or to instrument your application. This would add extra information to send along that would be useful for the MySQL Query Analyzer.

That said, from the feedback we are getting, MySQL Proxy works for most people"

I'm not sure I agree Arjen. The point is that with proxy I can switch stuff into and out of quan very quickly without affecting the setup or adding any code into the server itself. I'd rather have the overhead outside of the server than in it, and optionally available. That way I can look at a site and think - hmm, that should be running faster, swap it to quan for an hour or so, look at the results, fix any problems then swap it back to the standard server, all without having to start or stop the server, only having to modify the port that the app talks to.

I agree that it's nifty, and well done MarkM & team.
However, the proxy-based approach is problematic. Such instrumentation should be build-in or at least facilitated by the server.

Any extra tool, such as proxy, is an additional possible point of failure. And, in any case, overhead. In this case, I believe such overhead is unnecessary.

Another approach: I've seen USB RAID storage systems, like those in the following link:

http://www.usbgear.com/USB-RAID-System.html

I wasnt saying it was good or bad i was just wondering what other people thought of it i also hear places in china they put their new born in a bin if its a girl or a boy i cant rememember which one but it doesnt really matter because its still a baby.

You may want to read about King Herod and his use of slaughter of male children to remove any claim to the throne. Or about the Amazons. This quote could very well be references to both, although they are not the only such stories that abound in our rather bloody history. Remember that fiction and particularly fantasy and science fiction tend to use allegory to discuss disturbing trends in society in a way that is less confronting. Jonathan Swift's Gulliver's Travels was a social commentary as well as fantasy, for instance. If you turn it on its head, then it could very well be pointing to some of the more extreme elements that exist within our world at the moment. If it makes you think, it has done its job.

Nearly always different people will read different things into the same words by an author. In fact sometimes as your experiences change you will read different things into the same text at different times in your life. That is one of the joys of language and why I love well written works, regardless of the genre.

Thank you for some feeback its good to know people are reading this I'd just like to write a line from the book and people please react with what you think

"We seduce any prisoners of royal blood, then kill them to prevent any attempt to claim us or our offspring. Our daughters are raised to be warriors;our male progeny are slaughtered at birth." Keeping in mind this is fantasy i wont say how killing baby's on their gender is wrong i'd just like to see what other people think.

I don't think the intent was to confuse fact and fantasy. It was quite clearly fantasy all the way through, although I too had problems with the style switching between the central narrative and the journal extracts. Sometimes the best fantasy lies on the border of reality. This was exploited to great effect in, for instance, Rod Serling's "The Twilight Zone" series. I don't think it is lazy, far from it. It takes great skill to be able to take you through that journey in such a way that you keep wondering if it is possible. Unfortunately I don't think this was achieved, and as you pointed out the fantastic elements jarred against the ordinary.

Hate to double post but i dont think ive written down everything i have to comment on i think my previous post if it is accepted to be shown is a little harsh but i just wish certain books would know what type of book they are,Fantasy book's clearly state that they are a fantasy type book and the reader will take it as so but if your telling the reader this possibly is fact the reader (me) get's confused as i was perfectly fine imagining a futuristic version of the world today until they started bringing in phycics and goddesses with supernatural powers, at that stage i quickly made the change from fact to fiction but i was annoyed when it went back to boring normality then back to fantasy which is a lazy/uncreative way of writing because its basicly saying: yes this book is fantasy except it takes things in normal day life uses there name and purpose for the purpose of the book.Now im obviously not a writer but if i was going to create a trilogy i would rather spend 3 years writing on a excellent book then 1 year (more or less) on a fairly average book.In my opinion of course.Im really going on here but i think im forming a decent arguement.That arguement being why should i buy this book,What does this book have that seperates it from the other books?

hmm the book seems really wishy washy for me it stumbles shamelessly between fact theory and fantasy which isnt my idea of a good book umm it really can not decide where it is going to set up shop on the three plaza's being fact theory and fantasy and the review is right the book is rather alpha female so it doesnt appeal to me but im sure if you can stave off boardom and your a female then this book is excellent.I myself am 13 years of age and male so i think im being too much of a critic but im sure if i was female this book would be a excellent read.

You know Karen that I raised a glass in Gryphon's honour and also shed a few tears...and I am sending you hugs..

Having RAID over USB seems like a bad idea to begin with. I've tried both RAID1 and RAID5 over USB disks, and for some reason it always fails sooner or later. Checking the drives afterwards for badblocks reveals none. The RAID5 solution also tends to fail the rebuild when doing heavy reading (due to reconstruction).

The problem in your slow slow performance however might have something to do with /proc/sys/dev/raid/speed_limit_min&max being set to low. You can always check the status of your sync and thus the speed at /proc/mdstat. For me, upping the speed_limit_min to 100000 did the trick.

Fabrice, this has been going on for some weeks. As I mentioned before I never suspected a crawler responsible for the denial of service, and that may be a common problem. You may want to check your logs for the 19th of October onwards, as this is where the crawler is most virulent.

I'm not particularly interested in the number of complaints you've had. It is an irrelevant metric. It doesn't change the fact that I've had to protect my site from your crawler. I have logs showing the time, number of hits, and the correlation with server load. I'd suggest you stop trying to manage perceptions of the problem and just fix it.

Hi again,

Thanks for pointing out the domain name, we're going to look into what happened there - it's clearly not in our interests to annoy people so we always look into this kind of issues. To date (in more than a year) we have had only two complaints in total.

Cheers
Fabrice

A few things, Fabrice.

I am not Jan. It was a reference to a well known marketing campaign in Australia.

I never mentioned which of my sites was the target of the crawler. I happen to run quite a few on that particular server. As it happens 178 hits at the same time on one site (austcrimefiction.org) is hardly "twice a day". But it doesn't matter, I have your IP block banned and it will stay that way.

It doesn't matter how you try and justify what you do, as it happens you take my intellectual property without my knowledge or consent to make a profit. That at the very least doesn't sound fair, and if not a breach in law of the copyright act is certainly a breach in intent.

Hello Jan,

Sorry if our crawler has been a problem. I checked our system if we have your blog's feed in our records indeed, but the crawler shouldn't have visited it more than twice a day, and it only looks for new pages. Maybe you're referring to a different site?

As you said the purpose of our service is to highlight what people are saying about our clients and their competitors. Our clients in turn quite often try to engage with authors to clarify issues, run advertising campaigns, etc.

Regards
Fabrice

Sounds interesting. Sigh... another one for the list...

I just finished reading the American ARC of The Immortal Prince and it is only 503 pages. I'm wondering if some of the tedious dialog other posters didn't like was edited out. I found the world of Amyrantha quite compelling and could not put the book down until it was finished. I liked it so much that I will probably buy the foreign (I use that term knowing it is all a matter of perspective) editions of the sequels so I can read them without having to wait for them to be published in the U.S.

I love all of Jennifer Fallon's books, including the Tide Lord series. Gods of Amyrantha and Palace of Impossible Dreams have left me obsessed, and I will admit to being able to think of little else. I really enjoyed the concept behind the Immortal Prince, although it was quite dialogue heavy. I don't really like Arkady, especially in the latter books, but Declan and Cayal are to die for. I can't wait for the Chaos Crystal!