SQL/PostSQL/NoSQL

As one addicted to database technology since dBASEII, and having used just about everything since then and seen promising companies and their tech-children come and go, and also having read E.F. Codd, Chris Date, Fabian Pascal and Michael Stonebraker, to name a few, I like to think I know my way around relational databases, and a lot about pre-relational databases (dBASE, FoxPro, etc.), and a little about post-relational databases (Cache, etc.).

I've read a few things about NoSQL (technology or movement? That is the question!). As my colleague Stuart McLachlan rephrased the question: If it's a movement, the real question is whether it's a religious or bowel movement. The piece that prompted his excellent witticism can be found at:

http://radar.oreilly.com/2012/02/nosql-non-relational-database.html

The author does make a few good points, among them the difficulty of bending traditional SQL systems to handle massively scalable sub-second retrieval of the sort required for huge systems such as what might be found at Facebook, Amazon, Google, etc. Even in organizations much smaller than those, the problem remains. 

What seems to be ignored in this discussion, however, is the distinction between transactional databases (OLTP) and retrieval/analysis (OLAP) databases. In the former, accuracy and timeliness are of paramount importance. In the latter, timeliness takes second place to performance, and it is generally accepted that such systems will not deliver instantaneous accuracy, but rather snapshots of how the data looked at some previous moment. The granularity of the moments of interest is decided upon or settled for by some combination of hardware costs, need for close-to-now data, and so on.

I think that the NoSQL people blur this distinction purposely. In the case of Amazon, where there is a lot of buying and selling going on, the assumption is that inventory is, by and large, unlimited. Therefore the need for sub-second retrieval trumps the need for accuracy. Amazon can always re-order. 

That is not always the case. For several years, I was the database developer/DBA and programmer for a company that sold event-travel packages, and did deals with U2, Madonna, the Rolling Stones and so on. The basic scenario went like this: we bought several hundred tickets to every event in a tour, then bundled them with travel, hotel, car rentals and pre- and post-event parties. In this type of system, inventory is fixed. There are only on average 400 tickets available for any given event, and only so many hotel rooms, and so on. It is therefore paramount for the sales people in all four offices, plus the customers booking on the web site, to know exactly how many tickets remained in inventory, at all times. The acceptable time lag is essentially zero. 

The paean to NoSQL cited above ignores another critical factor: the rapid descent of solid state disks (SSDs), which are now available in sizes up to 10 terabytes (TB). A company called Fusion-io offers systems combining up to four of these SSDs for a total of 40 TB. That's large enough for lots and lots organizations with equal needs for performance and accuracy. The performance specs are awesome:

The 10 TB ioDrive Octal offers:

• More than 1.3 million IOPS with 6.7 GB/s bandwidth
• Extreme capacity density and lightweight footprint delivering up to 20 TB of ioMemory in a 1U server
• Process multiple terabytes of data in a fraction of the time
• Optimized performance for read-heavy environments such as data warehousing, scale-out architectures, supercomputing and research applications
• More work per unit of processing thanks to Fusion's cut-through architecture, which delivers performance increases as CPUs become more powerful
• Consolidate data-intensive infrastructure by an order of magnitude
• Reduce failure points and system complexity
• Reduce power and cooling costs of expensive, data-intensive server farms

Granted, these systems are not exactly cheap, but they are within the budget of many organizations. 

In short, rumours of the death of classic relational databases are premature. And in my opinion, the intellectual validity of the arguments in favour of RDMSes remains.

For more information about these SSDs, visit  www.fusionio.com. Regrettably, I own no shares in this company.

I do not mean that there is no place for NoSQL databases, but rather that their purpose is retrieval rather than transactions (Insert, Update, Delete). In fact, it could well be that the best system design might be a combination of the two approaches: use an OLTP database for transactions and a NoSQL database for retrievals. The problem remains, How and When to update the NoSQL database to reflect changes in the OLTP database. The answer to that question, I think, will depend on several factors: size of the databases, frequency of transactions, permissible lag time and so on.

A.

Income Made Outside the Country

Unless you don't read newspapers or their on-line equivalents, or tech-oriented websites (slashdot, etc.), you're probably aware of the New York Times piece and the ZDNet pieces on the subject of off-shore income.

Being a person rather than a multinational corporation, and further, being a Canadian who is neither a lawyer nor unacquainted with international law;

Being an actual person rather than a fictional person (this meaning the popular definition of corporations, which I might add conveniently places almost all the egregious behaviours of a fictional person without the realm of national laws), I must protest.

According to Canadian law, all income acquired outside Canada must be reported to our Department of Revenue. For example, a few years back I took a job in Bermuda as a DBA/SQL Developer, at a very nice rate, and Bermudian taxes on same were very low (4.5% if memory serves). But I also had to declare this income to Canada's Revenue people, since I maintained that my permanent residence was in Canada, and therefore Revenue Canada wanted their piece of the action.

Personally, I had no problem with this. It seems only reasonable to me that insofar as I maintain my permanent residence in Canada, then I am subject to the laws of Canada, regardless of where I happen to be contracting in another nation. These laws apply to my continuing status as a Canadian citizen. In the case of Bermuda, there are very strict laws about who is allowed to work there, and how long one is entitled to work there. Similar laws may apply to numerous other nations, or not. But that is not the point.

If I want to protect my Canadian nationality, then all foreign income must be reported and taxed accordingly. That's because I am an actual person, rather than a fictional person (i.e. a corporation). But if fictional persons want the status of "person", then IMO they ought obey the same rules as I.

I realize the significant consequences of this maxim to Canadian corporations: about 75% of the world's mining companies are headquartered in Canada. They make their phenomenal profits in Nigeria etc., and pay nothing to the Canadian government in the way of taxes on said income. The closer you look, the uglier it gets: the Canadian government actually co-funds housing projects and school-building projects in numerous domains, the better to serve those corporations' profits in the Third World -- at the expense of the local citizens.

I want the current Canadian law to apply equally to fictional persons as well as actual persons. And I would like USA law to reflect this policy (and not only the USA -- this is not a singular attack, but rather a plural attack, upon many corporations and nations).

The current target of attack is Apple, and justly so, IMO. But many, many other corporations are equally guilty of this egregious behaviour. To cite just one or two, look at Cisco and Intel and Nike and hundreds if not thousands of others.

According to Canadian law, if I make money off-shore, said income is subject to Canadian taxation on said income. All I am asking is that the same rule be applied to Fictional Persons, insofar as they maintain their headquarters in the USA or Canada or any member of the EU.

I'm asking more than that: the various unions involved have to grow up and realize that it's now One World, and demand that equal wages be paid to employees, regardless of the nation in which they work. I am cognizant of at least some of the effects of this policy:

• There would be no wage-edge in shipping the jobs to another nation. That is not to dismiss the edge in taxation that might be gained, but that is another issue, to be dealt with separately.
• The transportation costs of shipping good from China to the USA would far outweigh the advantages of shipping jobs off-shore, not to mention the temporal edge gained by local Just-In-Time shipping. Gant has proved this with its shirts.
• We might even get to the point where "Think global, buy local" becomes an economic mantra.
• Finally, this might cause rather abrupt upheavals in nations where the Powerful overwhelm the WeakerThans at every turn. I consider that a Good Thing.

I realize that this might cause various Canadian and USA and EU corporations to move their headquarters off-shore to escape local taxes, but that move would prove my point: these fictional persons are concerned only with profits, feeling no obligation to the nation in which they were fictionally born. The ultimate realization is that said corporations are no better than sluts, willing to go for the best best Cheap Trick their accountants can invent, and all this at the expense of locals in every nation.

To paraphrase Shakespeare, "Something (serious) is wrong in the state of Denmark."

Call me naive, but I tend to think of the planet as a city, of its nations as suburbs, and our situation as "We are all in this together", that every citizen of this planet ought to receive free medical care (admittedly a Canadian and Scandinavian concept, but gradually other nations are included), that all methods of income ought to be taxed equally (including capital gains, stock-market trading, real-estate flips and escalations, blue-chip coupons, and numerous other methods of income-accrual).

There remains one vague area in this mess: inheritance. This one, I agree, is problematic.
a) One ought to be entitled to disperse to one's descendants the wealth that one has accrued. I don't agree with is one, but numerous nations, including Canada, have agreed with this principle; The problem arises when certain families in certain nations own 90% of the wealth. In those nations, said accruals might amount to 70%+ of the national wealth,  GNP and GDP (c.f. the Phillippines, the USA, Canada and numerous other nations).
c) Numerous nations have adopted the policy that "What I own now is forevermore MINE". I do not accept this maxim. I can cite numerous nations in which small families own 90% + of their nations' wealth, and this is Wrong, Wrong, Wrong!

Under the current laws, I applaud the moves by Warren Buffet and William Gates, the former who admitted that he pays less taxes than his help, and the latter who has donated many billions of dollars toward causes such as, to name but one, the cure for malaria and AIDS medications.

These two people strike me as the models for behaviour of the massively wealthy persons in the world, and I salute them both. I stand at attention and salute them both.

I have no beef with accruing massive wealth. That is nothing more than a measure of one's influence upon the world. Steve Jobs and William Gates and the folks behind Linux and Wikipedia and WikiLeaks strike me as massive contributors toward this goal -- unlike, for example, Larry Ellison, whose massive wealth he chooses not to disperse to any of the world's people in need That is his current privilege, and according to current laws, that's perfectly OK. However I would argue that this evinces the problem with today's laws..

I wish that I were in their company, with billions to donate to worthy causes. Sadly, I am not. The best I could do is write some code and donate it to the world of database programmers, a minuscule contribution compared to theirs. But I don't have the funds to do what they do, so I do my tiny part.

What I want:
a)Fictional persons pay the same taxes as real persons do, meaning that all my off-shore income be subjected to the same taxes as if I made it in my native country. I want Apple etc. to obey the same laws.
b) "A Dollar Made is a Dollar Earned", no matter how it was made. Every stock trade, every real-estate flip, every rise in the value of one's stock-holdings, ought be taxed as if it were money made by working a job.The alternative is a society in which the worst possible thing you could do is hold a job, since you'll be caught holding the bag, every time.
That's my mantra.

Ultimately, I would argue for these positions as the true spirit of capitalism. Play by the rules or f**k off and demand your subsidies, all of which fly in the face of capitalism. And while I'm at it, To Hell with Bailing Out the Banks! None of them is too big to fail. Let them crumble just like the millions of real people whose lives crumbled under their weight. To think they are worth saving is to fall into the trap of "corporate socialism" -- socialism for the rich, and ugly unrepentant capitalism for the great unwashed.

I'm hoping that the Occupy movement is the first glimmer of a collective realization that the current arrangements demand a rethinking and even a revolution.

A.

Adele: Rolling in the Deep

I have no idea why this woman hits me so profoundly, but she does. I'd love to marry her, but given her track record, she'd be bound to write another album about the numerous ways in which I failed her. I'm willing to trade that for a year or so with her. She has boundless talent and she's beautiful and I'd do it in a New York moment.

Since the lyrics to "Rolling in the Deep" are publicly available, I'm going to copy them into this blog. IMO, they have all the bitterness and compression of classic ballads, and I'm glad that this song is not about me.

Following are the lyrics to "Rolling in the Deep":

There's a fire, starting in my heart

Reaching a fever pitch and it's bring me out the dark

Finally I can see you crystal clear

Go ahead and sell me out and I'll lay your ship here

See how I'll leave, with every piece of you

Don't underestimate the things that I will do

There's a fire starting in my heart

Reaching a fever pitch and it's bring me out the dark

The scars of your love reminds me of us

They keep thinking that we almost had it all

The scars of your love, they leave me breathless

I can't help feeling…

We could have had it all

Rolling in the deep

Your had my heart inside your hand

And you played it to the beat

Baby, I had no story to be told

But I've heard one of you and I'm gonna make your head burn

Think of me in the depths of your despair

Making a home down there as mine won't be shared

The scars of our love remind you of us

They keep me thinking that we almost had it all

The scars of your love, they leave me breathless

I can't help feeling

We could have had it all

Rolling in the deep

You had my heart inside your hand

And you played it to the beat

Could have had it all

Rolling in the deep

You had my heart inside your hand

But you played it with a beating

Through your soul through every open door

Count your blessings to find what you look for

Turn my sorrow into pleasured gold

You pay me back in kind and treasure what you sow

(Now I'm gonna wish you had never met me)

We could have had it all

(Tears are gonna fail, rolling in the deep)

We could have had it all

(Now I'm gonna wish you had never met me)

It all, it all, it all, it all

(Tears are gonna fall, rolling in the deep)

(Now I'm gonna wish you had never met me)

You had my heart and soul)

(Tears are gonna fall, rolling in the deep

And you played it to the beat

(Now I'm gonna wish you never had met me)

Could have had it all

(Tears are gonna fall, rolling in the deep)

Rolling in the deep

(Now I'm gonna wish you never had met me)

You had my heart and soul in your hand

(Tears are gonna fall, rolling in the deep)

But you played it, you played it, you played it to the beat

Adele, Love and Kisses! One day I hope to meet you, and kiss your hand. I'm running out of adjectives and superlatives to describe your impact upon me. I'm 3 times your age, but I continue to hope that one day I could say all this in person. I play this and many other of your songs daily. I'd hate to be on the bad end of your stick, but in trade for a few hours or days or even months with you, I'd accept that fate. Adele, you ROCK!

Maybe the thing I love best is how anti-cliche the YouTube "Rolling in the Deep" tracks are. Down with the era of "the only way to make it as a female musician" is by wearing scanty outfits and hiring sexually-provocative dancers. Up with sheer goddess talent!

Not to get too lofty here, but I can't resist quoting Arthur Schopenhauer: "Marry a woman for her conversational skills. After 40 years, that's all that shall remain." (Actually, I think that's a paraphrase, rather than an accurate quote, but you get the gist.)

The typical male trajectory is, (and perhaps it also applies to the female trajectory, but I daren't venture a position on that, despite a few exemplary cases, which are clearly insufficient evidence),

A maxim: "The thing that compels you to fall in love is precisely the thing you shall hate in a couple of years."

Case in point (one among many others, but shockingly frequent): she dressed provocatively when you first melted at the sight, then worked up the courage to approach her, then She said Yes, and before you knew it, She was your girlfriend. But she continued to dress provocatively, and your territorial instincts kicked in, and just like an outraged Father, you protested her going out in public dressed like that.

She: "Well, yeah, but isn't that what you loved about me in the first place?"
He: "Well, yeah, but now you're My Girl, and it's inappropriate."
She: "Oh, so now I'm supposed to be a virgin, without desires to be attractive?"
He: "I need you to be dressed 'Spoken For', else God knows who might start hitting upon you.
She: Shall I wear a burqa? Would that make you happy?"
He: "When you go out, your attire must say 'Spoken For', else I shall be insulted and ridiculed."
She: "In the 21st Century, it is men and their families who ought to pay the dowry, not the fathers of women! That is so 19th Century, and presupposes that women are required for nothing but childbirth. Welcome to the 21st Century. Men are obsolete. We can do it with test-tubes. So there!
1. We now have the vote, in at least some nations!"
2. Females outnumber males in every nation but China.

What has all this to do with Adele? Well, a lot. She strikes me as the logical consequent to Alanis Morrisette, whose "You Oughta Know" on her album "Jagged Little Pill" encapsulated all the bitterness of at least a generation of women, and at last count she sold about 5 million copies. Now it's Adele's turn, and she rocks!

A.

Memory Wall

This is the title of a collection of stories by Anthony Doerr. Mr. Doerr has received rave reviews from many places, which I shan't summarize here. Suffice to say that Mr. Doerr has rethought what it is to be a short (or rather longish) story. At 86 pages, the title story leaves the reader gasping for air; it has the strength and depth of novels three times its length. The word that comes to mind is Indelible, in the Dostoevsky sense (OK, I confess that Dostoevsky is my favourite novelist. I know, I audited second-hand a course taught by Vladimir Nabokov, then teaching "Russian Literature" at Cornell in Syracuse. Nabokov despised Dostoevsky and unofficially nominated Tolstoy to Sainthood.

His own books reveal his preference: everything is as perfect as Mozart. That, frankly, is why I prefer Dostoevsky: it's messy and hurried (no doubt due to impending gambling debts, augmented by the deadlines of serial-publication in newspapers).

To my taste, it is the hurry to get it down on paper that I love the most. Sentences and paragraphs are not so precisely chiseled as Tolstoy or Fowles or de Lillo would have carved them, and that is precisely why I love Dostoevsky. I get the feeling from his writing that he is rushing to capture Reality, as opposed to pausing and contemplating and shaping his sentences exquisitely. The latter approach shall always be outrun by Reality.

Dostoevsky reminds me most of Courbet, the French Realist painter. His works strike me as equally hurried: so many events and situations to capture, with so little time left.

Anthony Doerr has pretty much redefined our expectations of the short story form. OK, I'm making a judgement here, or at least stating a preference. Much as I admire Mavis Gallant and Alice Munro, I can't say that anything I've read by either has left me so shaken as the stories in Memory Wall. This book is going to live with me a long time.

A.

Massive Surveillance Project in Canada

The current issue (Feb. 2012) of small magazine published in British Columbia, Focus, contains a disturbing article by Rob Wipond, entitled Hidden Surveillance.

“This will revolutionize the way we police,” proclaimed Vancouver police in The Province, a Vancouver newspaper.


The technology is called Automatic License Plate Recognition (ALPR). I'm quoting Wipond's article:


The RCMP and BC government piloted ALPR in 2006 and have expanded it rapidly. BC now has 42 police cruisers equipped with the technology, including one with the Victoria Police Department (VicPD), one in Saanich, and two in our regional Integrated Road Safety Unit.  

Normally, area police manually key in plate numbers to check suspicious cars in the databases of the Canadian Police Information Centre and ICBC. With ALPR, for $27,000, a police cruiser is mounted with two cameras and software that can read licence plates on both passing and stationary cars. According to the vendors, thousands of plates can be read hourly with 95-98 percent accuracy. These plate numbers are automatically compared for “hits” against ICBC and Canadian Police Information Centre “hot lists” of stolen vehicles; prohibited, unlicensed and uninsured drivers; and missing children. When such “hits” occur, plate photos are automatically stamped with time, date, and GPS coordinates, and stored. The officer will investigate details in the above-mentioned databases directly, and may pull over suspect vehicles.

At least, that’s how the popular story goes, and it sounds wonderful. However, some news stories have quoted academics or civil rights advocates worried about what else this plate recognition technology is, or could be, used for. ALPR was developed by the British government in the 1990s to track movements of the Irish Republican Army. By 2007, the International Association of Police Chiefs was issuing a resolution calling for “all countries” to begin using ALPR and sharing population surveillance data for fighting gangs and terrorism. Today in the UK, ALPR is used for charging tolls, “risk profiling” travellers, and tracking or intercepting people using cars photographed near protests.

Mr. Wipond used the Canadian Freedom of Information Act to request some documentation pertaining to ALPR. In August, 2011 he received a 77-page document titled "Final Revision", and dated October 17, 2009. Wipond summarizes the document:

According to this document, the categories of people that generate alerts or “hits” in the ALPR system, alongside car thieves and child kidnappers, are much broader than has ever been disclosed publicly. And information on these people’s movements is being retained in a database for two or more years. For example, though you may not be stopped, your car is a “hit” and its movements are tracked and recorded if you’re on parole or probation or, in some cases, you’ve simply been accused of breaking a criminal law, federal or provincial statute, or municipal bylaw. You’re also a hit if you ever attended court to establish legal custody of your child, if you’ve ever had an incident due to a mental health problem which police attended, or if you’ve been linked to someone under investigation. The list of hit categories continues through three more pages, and a fourth page that the RCMP completely redacted.

This goes far above and beyond our expectations of "normal" police behaviour. In my opinion, this is a flat-out admission that the citizen is guilty: we just have to figure out, Guilty of What?

The article can be viewed at http://www.focusonline.ca/?q=node/312. If you wish, you can download the complete issue as a PDF file from the same page.

Arthur

Philosophy in High School

There's an interesting and provocative piece from the Boston Review, entitled Citizen Philosophers, Teaching Justice in Brazil, by Carlos Fraenkel. The article begins with a discussion with Almira Ribeiro, who teaches philosophy "in a high school in Itapuã, a beautiful, poor, violent neighborhood on the periphery of Salvador, capital of the state of Bahia in Brazil’s northeast. She is the most philosophically passionate person I’ve ever met," writes Fraenkel.


In 2008 a law was passed making the study of philosophy mandatory. As a result, about 9 million students take philosophy for 3 hours a week. The students learn something about engagement. 


“There are also other ways of political participation,” Ribeiro tells her students. She gives them the town hall’s phone number for complaints about infrastructure and asks them to find something in their street they want repaired. When one student calls, nothing happens. But when fifteen call, the city reacts. “You see that pothole?” she asks me. “It’s been closed. And that street lantern? It’s been fixed. Thanks to our philosophy class. . . . Politicians can’t afford disgruntled citizens who will vote them out of office.” 


You can read the whole piece at http://www.bostonreview.net/BR37.1/carlos_fraenkel_brazil_teaching_philosophy.php.

I definitely agree with the requirement to learn philosophy as a mandatory course in public schools. As it happened for me, I was somewhat ahead of the curve, but only because it wasn't taught, and so I had to seek it out and read. I began with Will and Ariel Durant's "The Story of Philosophy" when I was about 12. I was captivated by Diogenes, Parmenides, Socrates, Plato and later, by Hume, and much later, by Russell and Whitehead, and then Einstein and Heisenberg and Dyson. Whitehead had maybe the biggest impact upon me; "Process and Reality" most of all, but "Science and the Modern World" was pretty good, too.

I also took a couple of course in "Religious Studies" and learned a smattering about Buddhism, Shinto, Zoroastrianism. the I Ching and various other ancient beliefs and books. Lately I have been reading about Sikhism, and even visiting the local temple occasionally, not as a believer but as a disinterested investigator  Consequently, if I have any argument against this piece, it's the EuroCentrism.

Philosophy neither begins nor ends with the Western (nee Greek) tradition. Not to say that this is a dead-end. Truly exciting things are happening in the philosophy of science, cosmology, astrophysics and a few other sub-disciplines.

All that said, this remains a provocative piece.

Arthur

Homo mendax

Across the dining room table, seven-year-old Fiona has spread out a half-dozen dolls with whom she's acting out a long, elaborate story. Occasionally, others at the table are called upon to witness a fact, but really our presence is incidental--the story carries on whether we're at the table or not.

Aristotle and the scholastic philosophers called man the rational animal, homo rationale. John Stuart Mill spoke of homo economicus. Others called us homo reciprocans, reciprocal man.

Well, those are stories, too. How about man the storyteller? What would that Latin name be?

Homo mendax.

It makes politicians look a little less deviant?