In response to: On badly behaved crawlers

Comment from: Steve [Visitor]
Steve

Just to mention that we are now 2014 and magpie still hasn’t improved. My advice: ban it from your site right away. I get about 3 - 5 hits a second from that piece of junk when they decide to index my site and I’m tired of paying for their incompetence.

01/12/14 @ 19:21

In response to: MySQL 101 - Sorting and Searching: ORDER BY, WHERE, GROUP BY

Comment from: [Member]
aj

Hi David,

The JOIN statement creates a complex statement that builds on previous components. If you look at the first join (between author and book) you now have effectively a meta-table that includes fields from both. The next join needs only to reference fields from the meta-table, which in this case includes fields from book as well as author.

This is no to say that you couldn’t restructure the query to match your supposition, however you’d need to think carefully about the direction (LEFT/RIGHT) of the join as I’m using LEFT JOIN knowing that it will always provide me with a result provided there is a valid author, regardless of if that author has a book or publisher. If I structured the query such that book was before the FROM then I would need a valid book to list an author.

Note that I’m using the term meta-table as a convenience. The underlying database doesn’t need to create a separate structure.

11/08/14 @ 16:47

In response to: MySQL 101 - Sorting and Searching: ORDER BY, WHERE, GROUP BY

Comment from: David [Visitor]
David

Hello,

in the previous episode to this, a SELECT query was made with 3 joins, namely:

SELECT `title`,
CONCAT_WS(’ ‘,`author`.`first_name`,`author`.`last_name`) AS `author`,
`publisher`.`name` AS `publisher`,
`format`.`name` as `format`,
`price`, `isbn` FROM `book`
JOIN `author` ON `book`.`author` = `author`.`id`
JOIN `publisher` ON `book`.`publisher` = `publisher`.`id`
JOIN `format` ON `book`.`format` = `format`.`id`;

As I understand it, the tables `author`, `publisher`, and `format` each have a separate link back to the book table. Is that correct? Now in this episode you have the query:

SELECT CONCAT_WS(’ ‘,`author`.`first_name`,`author`.`last_name`) AS `author`,
`book`.`title` FROM `author`
LEFT JOIN `book` ON `author`.`id` = `book`.`author`
LEFT JOIN `publisher` ON `publisher`.`id` = `book`.`publisher`
WHERE `publisher`.`name` = ‘Bantam’;

I can see in the first join that the `book` table is linked back to the `author` table. But the second join to `publisher` is a join to the `book` table. I the thought that with multiple joins all tables following JOIN statements had to link back to the initital table referred to in the FROM statement.

08/08/14 @ 10:09

In response to: In memory of Eileen Mary Donnison (nee Cain)

Comment from: Boel [Visitor]
Boel

Adam, what a lovely tribute to your mother. She sounds like someone I’d like to have met. My condolences.

Boel

07/05/14 @ 09:50

In response to: In memory of Eileen Mary Donnison (nee Cain)

Comment from: Sarah [Visitor]
Sarah

Thank you for sharing this, she was an amazing Gran and an inspiring woman.

05/05/14 @ 22:28

In response to: MySQL 101 - Retrieving data: SELECT and JOIN

Comment from: [Member]
aj

Thanks, Jose, I’ve updated the links.

26/02/14 @ 18:33

In response to: MySQL 101 - Retrieving data: SELECT and JOIN

Comment from: Jose [Visitor]
Jose

Hey,

Great guides for starters (like me) only comment so far is about the website. The link for your ‘last episode’ has an extra http// in it (so it doesn’t work unless you remove it). And on your last episode (creating the database) you don’t seem to have a hyperlink to the next episode - just text saying what will be covered. Thought I’d comment just so people who are looking for these in order will have an easier time (I googled for this page to find it when the hyperlink on the previous wasn’t working).

Thanks again!
Jose

11/02/14 @ 02:53

In response to: MySQL 101 - Referential Integrity

Comment from: SQL dev [Visitor]
SQL dev

I have rarely seen programming taking advantage of Referential Integrity constraint e.g. CASCADE on UPDATE. It’s immensely useful specially in large application where data is subject to change by multiple people. Here is another good example of referential integrity in MySQL http://javarevisited.blogspot.com.ar/2012/12/what-is-referential-integrity-in-database-sql-mysql-example-tutorial.html which explains cascading.

09/01/13 @ 14:30

In response to: On badly behaved crawlers

Comment from: Neil [Visitor]
Neil

Just an update…We applied the robots.txt fix and noticed a distinct drop in origin traffic and CPU usage.

Not all magpie-crawler connections have gone but most have. We will only block the IPs as the last resort but it may come to that if we can’t get rid of them all via robots.txt.

Seems magpie is (as stated above) simply overly aggressive.

10/09/12 @ 19:36

In response to: Dealing with pre-emptive grief

Comment from: [Member]
aj

Hi Eli,

Unfortunately I ended up having to make the decision, as I knew I would. One thing you learn when owning pets is that you will outlive them, and that more often than not you are going to have to make some hard decisions. What you have to remember is that you have to act in the best interests of your pet. It is not betrayal to end suffering, although the point at which you decide that quality of life has gone will be different for different people and different situations. In fact, I see it as a duty, a responsibility of ownership.

I hope that your situation is easy on both of you, but please, don’t shy away from the hard decision if and when it becomes necessary. You will have doubts, that is only human - but if you love your dog the way it comes across from your words you will know when it is time to act.

30/08/12 @ 11:17

In response to: Dealing with pre-emptive grief

Comment from: Eli [Visitor]  
Eli

I took my twelve yr old golden retriever for a checkup and although she had been put on a reduced calorie diet and lost some weight and her arthritis was starting she had some really good days. The shock was the blood test showed she has leukemia and will die in 30 to 60 days. She shows no signs except the blood counts, eats well, no stomach problems but has anemia which slows her down. I am hoping I will not have to make that choice unless she stops eating or drinking as I feel I would feel I am betraying the animal that loves me most and will do anythings to please me. I understand how you feel pray so that he goes before you have to make a choice and he does not suffer.

30/08/12 @ 09:21

In response to: On badly behaved crawlers

Comment from: Neil [Visitor]
Neil

Hi all

The magpie crawler is also causing us some nasty issues on several large sites. The main issue for us is in fact that they persist in crawling non-existent pages, one of which alone is causing us over 20,000 404s per day.

We are taking steps now to ban magpie from our servers.

Cheers
Neil

24/08/12 @ 17:51

In response to: MySQL 101 - Creating your first database

Comment from: [Member]
aj

You are mistaken on a few fronts here. Firstly there is no error in the syntax, and the syntax used is ‘best practice’ for MySQL. Secondly, there is no single quote, they are backticks. Lastly it is not recommended to leave these out, as you set yourself up for problems in the future.

As to captcha - I’ll look into that.

30/07/12 @ 11:54

In response to: MySQL 101 - Creating your first database

Comment from: vaz [Visitor]
vaz

Note - the syntax that you used is wrong

You have mentioned : CREATE DATABASE “bookshop’;

Firstly - one can use either small oor caps for the create database

Secondly - if you give a single quote it will throw a ERROR 1064

Therefore, the correct syntax is as follows

create database sonic;
(no quotes)

Best….
(also one comment on the website caoptcha code - there is some problem, it refreshes the some captcha multiple times and does not allow to post, until one reloads the page and gets a new captcha. Also your captcha is little strange characters and one cant figure out the letters)

29/07/12 @ 06:36

In response to: On badly behaved crawlers

Comment from: Dave [Visitor]  
Dave

They’re being a nuisance for me as well. Whole subnet blocked now, for all servers.

15/03/12 @ 21:00

In response to: How to Write Crappy Code

Comment from: JC [Visitor]
JC

Here are some more:
Don’t indent your code, or better yet, indent it randomly.
Put inexplicable white space in here and there to make the file longer.
Always leave out of date ‘to-do’ comments in place, and never change a comment when you change the code.
If you must comment your fixes, be sure to leave out any meaningful references like CR or QC numbers, eg “// change (jc)”
Finally - and it’s not really a coding issue, but it does make for a good laugh - tell your boss that everything’s fixed and he can tell his client to relax, then check in the wrong version of the file!

29/11/11 @ 20:15

In response to: MySQL 101 - Replication

Comment from: Seppo Jaakola [Visitor]
Seppo Jaakola

Thanks for the writeup!

Galera Replication is missing in your reference list. Galera provides synchronous multi-master clustering for MySQL/InnoDB. And it is true multi-master, you can write to any row at any cluster node - just like MySQL/Cluster but works on InnoDB. And fully open source, downloads here: http://www.codership.com/downloads/download-mysqlgalera/

02/11/11 @ 21:06

In response to: MySQL 101 - Replication

Comment from: [Member]
aj

Thanks Giuseppe, I should have remembered that. If I’ve forgotten anyone else, please chime in with a comment.

01/11/11 @ 19:01

In response to: MySQL 101 - Replication

Comment from: Giuseppe Maxia [Visitor]
Giuseppe Maxia

Multiple masters replication, without the limitations found in MySQL, can be done easily using Tungsten Replicator (see http://code.google.com/p/tungsten-replicator/wiki/TRCMultiMasterInstallation#Multi-Master_Installation).

01/11/11 @ 18:53

In response to: From MySQL to SkySQL - Reflections

Comment from: [Member]
aj

Hi Henrik,

From my personal perspective I’d say that it isn’t a matter of sites being proactively shut down, but a “death of a thousand cuts". Ignoring the community sites is easy, they don’t provide any direct bottom-line benefit, they require resources that could be deployed in efforts that are more aligned to revenue generation, and they allow a platform for uncontrolled commentary. A company that is so tightly centralised as Oracle doesn’t have that sort of platform in their DNA. Unlike MySQL AB, which understood that the community would become their customer base, I’m not sure Oracle sees the benefit of Community. Users, yes, but users in Oracle’s thinking are paying customers.

We are already seeing signs of the lack of commitment to Community. Take a look at the forums, or MySQL Forge and tell me if you see signs of active engagement and a commitment to the community.

As to who should build a replacement and what it would be called, there are issues there. The name “MySQL” is a registered trademark, and if the site attracted a huge following you can bet the Oracle lawyers would take a dim view of any use of the trademark. However there are precedents and many products have community-run support sites that are not affiliated with the company producing the product. It could be done. But who should do it? It probably doesn’t matter much. As long as the future was assured and the site was unlikely to be dropped due to pressure from parties with their own agenda, who actually provides the platform is almost irrelevant.

The main thing required for any Community site is active moderation and management. I engaged with the community to put in place moderators for the MySQL forums, rather than have all of the moderation done by the ever-shrinking internal resources (the Community team was effectively decimated long before I left, and as mentioned, the web team was not doing much better). What I couldn’t do was give them the resources they needed to effectively take control of the site. I think that any new site should have community engagement and have a means by which the community can effectively manage the site. There are limits, of course, but making the site as much a community resource as possible while protecting it from misuse is a worthwhile although delicate balancing act.

20/10/11 @ 19:35

In response to: From MySQL to SkySQL - Reflections

Comment from: Henrik Ingo [Visitor]
Henrik Ingo

Thanks for sharing!

Suppose some community site is actually shut down in the future… In your opinion, what should we do then? Who should build a replacement site and what would it be called?

20/10/11 @ 18:32

In response to: MySQL 101 - Sorting and Searching: ORDER BY, WHERE, GROUP BY

Comment from: [Member]
aj

Francis, it is nearly impossible to answer the question without knowing more about your setup, and this probably isn’t the right forum. Millions of records should not be an issue, there are plenty of sites out there with many millions of records in MySQL databases not having issues with single table queries.

You really should first check that you are using indexes effectively. This is pre-empting a future episode, however you can use the EXPLAIN syntax to find out how the query is likely to use indexes. The episode “Creating your first database” does try to explain the use of indexes and why you need them. You might want to take another look at that.

If you want more information, there are plenty of places where you can get some interactive help. There is the #mysql channel on irc.freenode.net, and the official forums site at http://forums.mysql.com/.

13/09/11 @ 20:09

In response to: MySQL 101 - Sorting and Searching: ORDER BY, WHERE, GROUP BY

Comment from: francis tudlong [Visitor]  
francis tudlong

Hello i have this millions of records and im having problems with the retrieval of the data because its very slow. even for just fetching a single table what is the problem and what is the solution? any idea thanks in advance

13/09/11 @ 19:52

In response to: MySQL 101 - Retrieving data: SELECT and JOIN

Comment from: [Member]
aj

Thanks, Sheeri, a very good point. Hopefully I explained both the correct way and why the short version was used in that particular example, however you are correct, people will often just use the example code.

09/09/11 @ 23:09

In response to: MySQL 101 - Retrieving data: SELECT and JOIN

Comment from: Sheeri [Visitor]
Sheeri

I recommend naming fields in your INSERT. Not doing so saves keystrokes but at the expense of having your code break if you add a field.

Remember that folks will copy and paste your example, so you have to set a good one!

09/09/11 @ 21:51

In response to: Book Review: Overclocked - Cory Doctorow

Comment from: Taylor [Visitor]
Taylor

Wowzers, reading your comments about this book definitely wants to make me get it. I must admit that I’m not into SF books BUT I did read “Ender’s Game” a few years ago and I LOVED IT! It absolutely blew my mind. I had to read it for school and thank God I did or else I would have never stumbled upon it. I even told my younger brother to read it and he did. He also loved it. Either way. I might just have to trust you and get “Overlocked”

And I can’t believe you got it signed! And haha you read it in less than 24 hours. Wow, trooper!

-Taylor

23/06/11 @ 06:24

In response to: On badly behaved crawlers

Comment from: SergeiS [Visitor]  
SergeiS

I also had to block access via .htaccess

21/06/11 @ 20:09

In response to: On badly behaved crawlers

Comment from: Andrew [Visitor]
Andrew

Thanks for the tip, their “crawler” still misbehaves.

12/06/11 @ 01:06

In response to: How to Write Crappy Code

Comment from: George Nagy [Visitor]
George Nagy

Great rant! I’ve run across similar code and want to add one more way to write crappy code: Use meaningless file names!

I’ve had to work on code bases with names like leftFish.inc.php (And not on code that had anything to do with aquariums either :( )

21/05/11 @ 03:41

In response to: The First Day of the Rest of My Life

Comment from: Max Mether [Visitor]
Max Mether

Welcome Adam!

04/04/11 @ 17:59

In response to: The First Day of the Rest of My Life

Comment from: Ivan Zoratti [Visitor]
Ivan Zoratti

Adam,

A very warm welcome!
I am sure you will enjoy working at SkySQL!

Let’s make our first days a great and enjoyable experience to share with all!

-ivan

03/04/11 @ 18:52

In response to: The First Day of the Rest of My Life

Comment from: Kaj Arnö [Visitor]
Kaj Arnö

Welcome Adam!

03/04/11 @ 16:39

In response to: On badly behaved crawlers

Comment from: Thomas [Visitor]
Thomas

The exact same problem. Their entire range 94.28.34.192/26 has been blocked from all of my servers. Leeches!

26/12/10 @ 23:34

In response to: Climate change and feeling helpless

Comment from: [Member]
KC

I’ve managed to cut off some of the flow into the chook yard and get some drainage out the other side so it’s still a part bog / part lake - but not quite as bad as it was.

But the wind has come up now and they are talking possible destructive - but you wouldn’t bloody credit it - I’m up to my ankles in water and the bloody sun’s just come out!

Weather bureau is talking possible thunderstorms - wouldn’t be surprised it’s 15 degrees outside and with that sun out - sheesh.

That big redgum in the soak - is now in the middle of a large fast running stream! I’ve got some photos - should have taken the video camera though - might try and get down there after a shower and a coffee.

04/09/10 @ 13:07

In response to: There has to be a better way

Comment from: Evan [Visitor]
Evan

And these issues are tripled if you have to mess with the DNS for them!

13/10/09 @ 03:11

In response to: MySQL 5.1 - Pssst! The secret is out!

Comment from: Eric Herman [Visitor]
Eric Herman

Some of the developers of the MySQL Enterprise Monitor have been running 5.1 for a year or more. For us this had more to do with validating that we were ready than validating the mysqld – for our uses, the 5.1 mysqld been solid for a long time. Me? I switch around, and even play with 6.x sometimes; speaking of, I need to give 6.0.8-alpha a whirl ….

19/01/09 @ 18:33

In response to: Book Review: Do Androids Dream of Electric Sheep? - Philip K. Dick

Comment from: Gene [Visitor]
Gene

What does Philip K. Dick wishes to say about the american society in 1968? can anyone tell me? I am doing a research of the book..

08/12/08 @ 23:07

In response to: MySQL Query Analyzer - first impressions

Comment from: Mark Callaghan [Visitor]  
Mark Callaghan

It it great that this won’t require the proxy because that would limit its use.

I can’t restart apps to use a different port to talk to the server. I expect many of us have that constraint.

I wouldn’t put the proxy in front of a critical server. That is just one more point of failure. And I have yet to see dbt2 or sysbench results that include the use of deployment. Is the proxy still single-threaded? Does it run Lua on every query and result?

20/11/08 @ 00:04

In response to: MySQL Query Analyzer - first impressions

Comment from: [Member]
aj

Just on getting data from Proxy, have you seen the interview with Mark on the DevZone?

Take a look down towards the bottom at the answer to the question “Are you exploring ways to use MySQL Query Analyzer without needing Proxy middleware?” where Mark states:

“We don’t want MySQL Proxy to be the only source of the metrics we are collecting. How you get this data into the Service Manager is pretty flexible and is based on REST. There are some other sources we want to explore. We would like to consume log data, we would like to consume the slow query log, or the general query log, even if it is sampled from time to time, so that you don’t have to use MySQL Proxy to get this information.

Also, there are ways to instrument the MySQL Connectors to get this information, or to instrument your application. This would add extra information to send along that would be useful for the MySQL Query Analyzer.

That said, from the feedback we are getting, MySQL Proxy works for most people”

19/11/08 @ 16:18

In response to: MySQL Query Analyzer - first impressions

Comment from: [Member]
aj

I’m not sure I agree Arjen. The point is that with proxy I can switch stuff into and out of quan very quickly without affecting the setup or adding any code into the server itself. I’d rather have the overhead outside of the server than in it, and optionally available. That way I can look at a site and think - hmm, that should be running faster, swap it to quan for an hour or so, look at the results, fix any problems then swap it back to the standard server, all without having to start or stop the server, only having to modify the port that the app talks to.

19/11/08 @ 15:44

In response to: MySQL Query Analyzer - first impressions

Comment from: Arjen Lentz [Visitor]  
Arjen Lentz

I agree that it’s nifty, and well done MarkM & team.
However, the proxy-based approach is problematic. Such instrumentation should be build-in or at least facilitated by the server.

Any extra tool, such as proxy, is an additional possible point of failure. And, in any case, overhead. In this case, I believe such overhead is unnecessary.

19/11/08 @ 15:04

In response to: USB RAID on Linux fails to live up to expectation

Comment from: Evan [Visitor]
Evan

Another approach: I’ve seen USB RAID storage systems, like those in the following link:

http://www.usbgear.com/USB-RAID-System.html

12/11/08 @ 17:54

In response to: Book Review: The Dragon Queens - Traci Harding

Comment from: joshua [Visitor]  
joshua

I wasnt saying it was good or bad i was just wondering what other people thought of it i also hear places in china they put their new born in a bin if its a girl or a boy i cant rememember which one but it doesnt really matter because its still a baby.

06/11/08 @ 19:07

In response to: Book Review: The Dragon Queens - Traci Harding

Comment from: [Member]
aj

You may want to read about King Herod and his use of slaughter of male children to remove any claim to the throne. Or about the Amazons. This quote could very well be references to both, although they are not the only such stories that abound in our rather bloody history. Remember that fiction and particularly fantasy and science fiction tend to use allegory to discuss disturbing trends in society in a way that is less confronting. Jonathan Swift’s Gulliver’s Travels was a social commentary as well as fantasy, for instance. If you turn it on its head, then it could very well be pointing to some of the more extreme elements that exist within our world at the moment. If it makes you think, it has done its job.

Nearly always different people will read different things into the same words by an author. In fact sometimes as your experiences change you will read different things into the same text at different times in your life. That is one of the joys of language and why I love well written works, regardless of the genre.

06/11/08 @ 18:39

In response to: Book Review: The Dragon Queens - Traci Harding

Comment from: joshua [Visitor]  
joshua

Thank you for some feeback its good to know people are reading this I’d just like to write a line from the book and people please react with what you think

“We seduce any prisoners of royal blood, then kill them to prevent any attempt to claim us or our offspring. Our daughters are raised to be warriors;our male progeny are slaughtered at birth.” Keeping in mind this is fantasy i wont say how killing baby’s on their gender is wrong i’d just like to see what other people think.

06/11/08 @ 15:59

In response to: Book Review: The Dragon Queens - Traci Harding

Comment from: [Member]
aj

I don’t think the intent was to confuse fact and fantasy. It was quite clearly fantasy all the way through, although I too had problems with the style switching between the central narrative and the journal extracts. Sometimes the best fantasy lies on the border of reality. This was exploited to great effect in, for instance, Rod Serling’s “The Twilight Zone” series. I don’t think it is lazy, far from it. It takes great skill to be able to take you through that journey in such a way that you keep wondering if it is possible. Unfortunately I don’t think this was achieved, and as you pointed out the fantastic elements jarred against the ordinary.

05/11/08 @ 16:16

In response to: Book Review: The Dragon Queens - Traci Harding

Comment from: joshua [Visitor]  
joshua

Hate to double post but i dont think ive written down everything i have to comment on i think my previous post if it is accepted to be shown is a little harsh but i just wish certain books would know what type of book they are,Fantasy book’s clearly state that they are a fantasy type book and the reader will take it as so but if your telling the reader this possibly is fact the reader (me) get’s confused as i was perfectly fine imagining a futuristic version of the world today until they started bringing in phycics and goddesses with supernatural powers, at that stage i quickly made the change from fact to fiction but i was annoyed when it went back to boring normality then back to fantasy which is a lazy/uncreative way of writing because its basicly saying: yes this book is fantasy except it takes things in normal day life uses there name and purpose for the purpose of the book.Now im obviously not a writer but if i was going to create a trilogy i would rather spend 3 years writing on a excellent book then 1 year (more or less) on a fairly average book.In my opinion of course.Im really going on here but i think im forming a decent arguement.That arguement being why should i buy this book,What does this book have that seperates it from the other books?

05/11/08 @ 15:58

In response to: Book Review: The Dragon Queens - Traci Harding

Comment from: joshua [Visitor]  
joshua

hmm the book seems really wishy washy for me it stumbles shamelessly between fact theory and fantasy which isnt my idea of a good book umm it really can not decide where it is going to set up shop on the three plaza’s being fact theory and fantasy and the review is right the book is rather alpha female so it doesnt appeal to me but im sure if you can stave off boardom and your a female then this book is excellent.I myself am 13 years of age and male so i think im being too much of a critic but im sure if i was female this book would be a excellent read.

05/11/08 @ 15:45

In response to: Vale Gryphon - Goodbye old friend

Comment from: Patricia [Visitor]  
Patricia

You know Karen that I raised a glass in Gryphon’s honour and also shed a few tears…and I am sending you hugs..

04/11/08 @ 10:09

In response to: USB RAID on Linux fails to live up to expectation

Comment from: Mattijs [Visitor]  
Mattijs

Having RAID over USB seems like a bad idea to begin with. I’ve tried both RAID1 and RAID5 over USB disks, and for some reason it always fails sooner or later. Checking the drives afterwards for badblocks reveals none. The RAID5 solution also tends to fail the rebuild when doing heavy reading (due to reconstruction).

The problem in your slow slow performance however might have something to do with /proc/sys/dev/raid/speed_limit_min&max being set to low. You can always check the status of your sync and thus the speed at /proc/mdstat. For me, upping the speed_limit_min to 100000 did the trick.

01/11/08 @ 11:36