Tuesday, May 9, 2017

Use records to improve readability and flexibility of your code

Suppose I've created a table to keep track of hominids:

   hominid_name     VARCHAR2 (100),
   home_territory   VARCHAR2 (100),
   brain_size_cm    INTEGER

I might then write code like this:

   l_b_hominid_name    VARCHAR2 (100) := 'Bonobo';
   l_b_brain_size_cm   INTEGER := 500;
   l_g_hominid_name    VARCHAR2 (100) := 'Gorilla';
   l_g_brain_size_cm   INTEGER := 750;
   l_n_hominid_name    VARCHAR2 (100) := 'Neanderthal';
   l_n_brain_size_cm   INTEGER := 1800;

What do you think?

I find the little voice of Relational Theory inside my head rebelling.

"All that repetition! All that denormalization! All that typing (or copy-pasting, which is even worse)!"

Surely if I should avoid having redundant data in rows of my tables, I should avoid redundant code, too?

Yes, I should.  I don't like to see long lists of declarations, especially when the names are very similar and follow a pattern. 

A fine way to avoid this kind of code is to use record types to group related variables together within a named context: the record variable. So I could rewrite the declaration section above to:

   l_bonobo        hominids%ROWTYPE;
   l_gorilla       hominids%ROWTYPE;
   l_neanderthal   hominids%ROWTYPE;
   l_bonobo.hominid_name := 'Bonobo';
   l_bonobo.brain_size_cm := 500;
   l_gorilla.hominid_name := 'Gorilla';
   l_gorilla.brain_size_cm := 750;
   l_neanderthal.hominid_name := 'Neanderthal';
   l_neanderthal.brain_size_cm := 1800;

Notice that I now move the initializations of the variable (well, record.field) values to the executable section. That's because PL/SQL does not yet offer a built-in function (in object-oriented lingo, a constructor method) for record types.

So I no longer have six declarations - just three. And, of course, if my table had 15 columns and I had declared a separate variable for each of those, I would have been able to shrink down my declarations from 45 to 3!

Still, I don't like putting all that initialization code in the main body of my block. How about if I create my own "record constructor" function, and then call that:

   name_in IN hominids.hominid_name%TYPE,
   home_territory_in IN hominids.home_territory%TYPE,
   brain_size_cm_in IN hominids.brain_size_cm%TYPE)
   RETURN hominids%ROWTYPE
   l_return hominids%ROWTYPE;
   l_return.hominid_name := name_in;
   l_return.home_territory := home_territory_in;
   l_return.brain_size_cm := brain_size_cm_in;
   RETURN l_return;

   l_bonobo        hominids%ROWTYPE := new_hominid ('Bonobo', NULL, 500);
   l_gorilla       hominids%ROWTYPE := new_hominid ('Gorilla', NULL, 750);
   l_neanderthal   hominids%ROWTYPE := new_hominid ('Neanderthal', NULL, 1800);
   DBMS_OUTPUT.put_line (l_neanderthal.brain_size_cm);

Ahhhhh. Just three declarations. Default values assigned in the declaration section. All the details of the assignments hidden away behind the function header.

And when I add a new column to the table (or generally a field to a record), I can add a parameter to my new_hominid function, along with a default value of NULL, and none of my existing code needs to change (unless that new column or field is needed).

Yes, I like that better.

How about you?

Wednesday, May 3, 2017

Getting my Oracle Database 12c Release 2 up and running on Mac via Docker

I love to follow in the footsteps of people who are braver, smarter and more knowledgeable than me.

So I was happy to wait till SQL Maria (Maria Colgan) published her blog post on Oracle Database 12c now available on Docker, with step-by-step instructions for taking advantage of the new Docker image for 12.2 now available (specifically, 12.2 via Docker on Github, 12.2 via Docker at the Docker Store).

I am happy to report that I can now connect SQL Developer to my containerized 12.2 database. Thank you, Maria, for a very helpful post!

Now, I am not going to repeat everything Maria already wrote. That would be silly. I will simply point out some things you might find helpful as you do the same thing I did (follow in Maria's footsteps - which, literally, meant lots of copy-pasting rather dumbly).

1. Watch out for those dashes when you copy/paste.

Docker was not responding as expected to my commands and I (well, actually, Gerald) eventually noticed that the dash, copied from the blog post, was too long - it had been translated into a different character. So watch out for that! You might need to retype the command yourself.

I hate that.


2. Create your own folder for your Oracle Database files. I know it should be obvious. But I am a copy-paste sorta guy, and probably the only one in the world who would copy this command into my terminal and expect it to work:

docker run --name oracle -p 1521:1521 -p 5500:5500 
-v /Users/mcolgan-mac/oradata:/opt/oracle/oradata 

And it did - once I created my own folder for the files, and replaced that in the command.

Oh and by the way, that entire command (once you swap out mcolgan-mac for your own foler) needs to be one one line.

After that, everything went very smoothly and, again following Maria's wonderfully clear steps, I had my database up and running.

Then I set up my connection in SQL Developer:

and voila! My own 12.2 database running in a Docker container, on my Mac.

Thanks, Maria!
Thanks, Gerald!
Thanks, Docker!
Thanks, Oracle!

Monday, May 1, 2017

Deterministic functions, caching, and worries about consistent data

A developer contacted me with the following questions last week:

We have created a function that returns a single row column value form a query. When we call this function with the same input values it takes to long to return. Example:

select max (det_function('A2')) from dual connect by rownum <= 1000000

But when we change the function to a deterministic function the statement returns really fast. The only thing where we are unsure is what happens when the tables has changed to which the statement of the function selects? Do we need a to commit this table to bring oracle to re-execute the statement in the function and not use the cache or what should we do to get a consistent return value?

   v_ident   NUMBER;
     FROM my_table
    WHERE id = v_in_id;

   RETURN v_ident;

A function is deterministic if the value returned by the function is determined entirely by its input(s).

The following function, for example, is deterministic:

FUNCTION betwnstr (
   string_in      IN   VARCHAR2
 , start_in       IN   INTEGER
 , end_in         IN   INTEGER
   RETURN (SUBSTR (string_in, start_in, end_in - start_in + 1));
END betwnstr;

You can also quickly see, I hope, that any function that contains SQL (like the first function defined above) cannot possibly be deterministic: it depends on the contents of one or more tables to do its job, and those datasets are not passed as IN parameters.

Does that mean the compiler will complain? No! But it does mean that you could create real problems for yourself if you are not careful about your use of this keyword.

So the rule should be: Only add the DETERMINISTIC keyword to truly deterministic functions.

Why? Why should it matter? Because under certain circumstances (such as the one identified by the developer above), Oracle Database will not execute your function, but instead simply use a previously cached return value.

Within the scope of a single server call (e.g., execution of a PL/SQL block), Oracle Database will keep track of input and return values for your deterministic functions. If in that same server call, you pass the same input values to the function, the database engine may choose to not actually execute the function, but instead simply pass back the previously-computed return value (for those same inputs).

That's why this developer saw such a great leap forward in performance.

Once that SELECT statement finishes, though, memory for the cache is released. When and if that same query is run again, the engine will start rebuilding and using that cache.

While that statement executing, though, no matter what sort of changes are made to the table, no matter if a commit is issued or not, those changes will not be visible to the statement that called the function.

That's why I will repeat The Rule again:

Only add the DETERMINISTIC keyword to truly deterministic functions.

If your function contains a SELECT statement and you want to call it from a SELECT statement, the best thing to do is take the SQL out of the function and "merge" it into your SQL - in other words, no user-defined functions. Just SQL.

Rob van Wijk offers lots more details on the behavior and performance of deterministic functions here. You will also be well-served to read Bryn Llewellyn's in-depth exploration of How to write a safe result-cached function.

Rather than repeat all those findings, I will simply conclude with:

1. Use the DETERMINISTIC function primarily as a way to document to future developers that your function is currently free of side effects, and should stay that way.

2. If you are looking for ways to improve the performance of functions executed inside SQL, learn more about the UDF pragma (new in Oracle Database 12c Release 1).

3. See if the function result cache feature (also explored in Bryn's blog post) might be applicable to your situation.

4. Do not call user-defined functions from SQL statements that in turn contain SQL statements (or at least do so with extreme caution). That SQL inside the function is not part of the same read-consistent image as the data set identified by the "outer" SQL.

Thursday, April 20, 2017

Tips for getting along with your DBA

Developers and DBAs: can't we all just get along?

Sure we can!

We just have to break out of the old routine of

Developer: Hey, DBA, add twelve indexes to make my code run faster!
DBA: Hey, Developer, tune your code to make it run faster!

That is, finger-pointing.

Instead, we need to work together, and developers I am not the least big reluctant to say:

It's up to us, not the DBAs, to take the first steps.

So here are tips on what you, the developer, can do to foster a strong, collaborative and highly productive relationship with your DBA:

1. Ask your DBA for advice. 

"I want to make my code run faster. What do you think I should do?" There's no better to improve a relationship than to show some humility and express interest in the opinions - and knowledge - of others.

2. Do the right thing. 

Learn about the performance-related features of PL/SQL (and SQL) and apply them. Here are some links to help get started:

PL/SQL Optimization and Tuning (Doc)
High Performance PL/SQL Videos
SQL Analytics Videos by Connor McDonald
Introduction to Indexing Videos by Chris Saxon

3. Give your DBA a heads-up when your pattern of writing code changes. 

Utilizing new and different features of PL/SQL can have a ripple effect on memory consumption and overall application performance. Don't blindside your DBA.

For example, you learn about executing "bulk SQL" from PL/SQL. So cool! So powerful! And potentially a big PGA memory suck, through the use of collections.

Or you discover the Function Result Cache. Another very exciting enhancement added in 11.1. "Hey, I'm going to add the RESULT_CACHE clause to 100 functions. So easy!" Yes, but you might kill overall database activity with latch contention.

Friday, April 14, 2017

Databases for Developers class on Oracle Dev Gym: take it any time!

Chris Saxon, a Developer Advocate at Oracle, has put together a 12 week "bootcamp" introduction to SQL. Each week consists of a short video, plus 3 quizzes. You probably won't need more than 30 minutes to complete them. Every fourth week, Chris will be on a live webcast to answer questions.

While there is a start date for each new class, there is no end date.

This means that you if you missed the beginning week or two (or seven!), you can still register for the class and take the earlier classes. To do this, visit the Oracle Dev Gym (for which you will need an Oracle account). Then go to the Classes page in one of two ways, shown below with the blue arrows.

Click on Databases for Developers, then click on the Register button.

You can then pick from any of the weeks in the Course Outline that have already been started, and work your way through them.

We hope you enjoy, and get lots out of, the Databases for Developers course! And while you are at the Dev Gym, be sure to check out our weekly tournaments as well our library of over 2,500 quizzes on SQL, PL/SQL, Database Design, Oracle Application Express and Logic!

Thursday, April 6, 2017

Does level 3 optimization change PL/Scope data? No!

I gave a webinar on April 6, 2017 for the Taste of Kscope17 series for ODTUG (odtug.com) on Change Impact Analysis with PL/Scope. Here are the slides from SlideShare. I will add a link to the video when it is available.

After my presentation, this question came up: if you set the optimization level to 3 (inlining of subprogram code), will that change the PL/Scope data gathered? Interesting question.

Suppose your function body contains an assignment to variable x. Just that one place. But the function is called in ten places. Will PL/Scope find ten assignments to x or just one?

Just one, as you can see in this LiveSQL script. The identifier information is gathered before optimization. Which makes perfect sense. Post-optimized code is no longer PL/SQL code.

Here's the procedure I tested this one:


      RETURN p * 10;


      RETURN p * 10;
   DBMS_OUTPUT.put_line (f1 (1));
   DBMS_OUTPUT.put_line (f2 (TRUE) + f2 (FALSE));

   PRAGMA INLINE (f3, 'NO');
   DBMS_OUTPUT.put_line (f3 (55));

Here's the query I used to get my identifier information back out.

  SELECT i.signature ||'-'|| s.line ||'-'|| s.text text 
    FROM    user_identifiers i 
            user_source s 
         ON (    s.name = i.object_name 
             AND s.TYPE = i.object_type 
             AND s.line = i.line) 
   WHERE object_name = 'PLSCOPE_DEMO'  
ORDER BY s.line

And the output is the same regardless of the optimization level:

47BFC756469F1D97B6C84EF73A9C5D48-5-   FUNCTION f1 (p NUMBER)
785705602C9312732B24D9A341360ACF-5-   FUNCTION f1 (p NUMBER)
C762ED3C314ABB3D7257031401DD1583-5-   FUNCTION f1 (p NUMBER)
C762ED3C314ABB3D7257031401DD1583-5-   FUNCTION f1 (p NUMBER)
2C17DB6428F739B212C1E11EED057D63-6-      RETURN PLS_INTEGER
785705602C9312732B24D9A341360ACF-9-      RETURN p * 10;
1B5895A65C6952FDD192221BFC45A132-12-   FUNCTION f2 (p BOOLEAN)
C0BA9F319D1E759AD02A3C7138D62232-12-   FUNCTION f2 (p BOOLEAN)
EE1C5F13825B7DF0BF06D82DE633992E-12-   FUNCTION f2 (p BOOLEAN)
1B5895A65C6952FDD192221BFC45A132-12-   FUNCTION f2 (p BOOLEAN)
2C17DB6428F739B212C1E11EED057D63-13-      RETURN PLS_INTEGER
C0BA9F319D1E759AD02A3C7138D62232-16-      RETURN CASE WHEN p THEN 10 ELSE 100 END;
335958BDCA260D5E550A47F8194048DC-19-   FUNCTION f3 (p PLS_INTEGER)
205AC954A1ADD2BC3DC6A112A2081ADA-19-   FUNCTION f3 (p PLS_INTEGER)
2C17DB6428F739B212C1E11EED057D63-19-   FUNCTION f3 (p PLS_INTEGER)
335958BDCA260D5E550A47F8194048DC-19-   FUNCTION f3 (p PLS_INTEGER)
2C17DB6428F739B212C1E11EED057D63-20-      RETURN PLS_INTEGER
205AC954A1ADD2BC3DC6A112A2081ADA-23-      RETURN p * 10;
C762ED3C314ABB3D7257031401DD1583-26-   DBMS_OUTPUT.put_line (f1 (1));
1B5895A65C6952FDD192221BFC45A132-29-   DBMS_OUTPUT.put_line (f2 (TRUE) + f2 (FALSE));
1B5895A65C6952FDD192221BFC45A132-29-   DBMS_OUTPUT.put_line (f2 (TRUE) + f2 (FALSE));
335958BDCA260D5E550A47F8194048DC-32-   DBMS_OUTPUT.put_line (f3 (55));

For more information about PL/Scope:

For more information about Inlining:

Sunday, April 2, 2017

Wikileaks bombshell: PL/SQL source of all other modern programming languages!

Copyright @2017 CodeNewsWire "News coders can use", dateline 1 April 2017

Wikileaks dumped its latest batch of revelations on the world on April 1, 2017, this time focusing on the world of software programming. From dishing out the dirt on the origins of the Internet (think: Area 51) to emails candidly deriding JSON as nothing more than the latest attempt (XML being the last one) to avoid carefully designing your database, this trove of previously secret secrets is sure to keep Silicon Valley gossiping for months.

But buried deep within the 2.5 trillion byte download is evidence of a conspiracy so vast, so unbelievable, so extraordinary, that it is hard, well, to believe.

But if it came from Wikileaks it must be true. And that conspiracy was built around - and is maintained around - this incredible bit:
All modern programming languages, from Java to JavaScript, Scala to Go, are actually all implemented in the Oracle PL/SQL language. Oh, and Linux, too.
You are probably laughing to yourself, right now, right? PL/SQL? That straightforward - and some might argue, rather archaic - procedural language, apparently useful only for managing transactions in the Oracle Database? How could you possibly implement Java in it? Linux? JavaScript?

Wikileaks has, apparently, two words for you:

It is well-known to practitioners of PL/SQL that there are several documented indeterminate behaviors in the language (which some, cynically, try to brush aside as merely "undocumented"). For example, the state of a variable that you SELECT INTO will be indeterminate if the statement raises TOO_MANY_ROWS. It seems to usually have the data from the first row selected in it, but this cannot be trusted.

Developer responses in the modern age (aka, the Age of Apps) to this indeterminacy have been to shrug and get on with life.

But Wikileaks has discovered minutes of a secret meeting taking place in 1991 in the office of the CEO of Oracle, attended by none other than James Gosling (creator of Java), Linus Torvald (inventor of Linux and Git), Brendan Eich (creator of JavaScript) and several others whose identity were masked in the minutes.

At this meeting, Larry Ellison disclosed that his engineers had designed PL/SQL to exploit quantum entanglement (which manifest as "indeterminacies") as a pathway into multiverse threading. The result was a programming language so elegant, so powerful, so subtle and so mysterious that it can be used to implement anything and everything.

The assembled experts were blown away. And thoroughly convinced by a 5 minute demonstration by Ellison, which involved, among other things, using PL/SQL to look into the box containing Schroedinger's Cat to tell us precisely and unambiguously whether or not it is alive. Or was. Or could be. Whatever.

The fear from the crowded meeting was evident, but Ellison put those fears to rest. "Don't worry, fellas," he was recorded as telling them. "We are not going to announce this news to the world. It will be too destabilizing. Instead, we've built a quantum-level API that you can all use to build whatever you want. And if you insist on continuing to use C, that's OK, too, because we've used the PL/SQL tachyon exploit to travel back in time and re-implement C in PL/SQL as well."

In the end, all these language experts agreed: there was too much to gain from PL/SQL to ignore it. But the world could never know. And so it was decided: Oracle would continue to promote PL/SQL as a database programming language, special-purpose and not very object-oriented. Purveyors of other languages would continue to make fun of PL/SQL and tout their own latest and greatest innovations.

CodeNewsWire reached out to Edward Snowden, whistleblower supreme (or arch-traitor, depending on your point of view), regarding this incredible revelation. "What?" he replied. "This is news? I thought that was in my dump from the NSA and CIA. All the best Agency developers write nothing but pure PL/SQL, on hopped-up quantum computers. And they use edition-based redefinition."

Steven Feuerstein, author of way too many books on PL/SQL, was hit harder by this news than most. On the one hand, he was pleased to hear about the enhanced power of PL/SQL. On the other hand, as he expressed it on his Twitter account, "How could I have missed something as big as this? And could I get another book out of it?"

Determined to gain insight into what really went on - and is going on, and will go on, all at the same time - he tracked down Linus Torvald to a heavily fortified Git Repo in a Helsinki rave club. Torvald wouldn't open the door, but he did shout the following: "Go away! Linux is mine, all mine! I was never at that meeting! The cat is dead, always dead, in all the universes I've visited using my PL/SQL transporter. Oh, crap."

Now the world knows.

It's all PL/SQL, all the time.

But don't worry, you can live in denial, and keep on programming in JavaScript or Go or Went or Ruby or Scala or Java.

Just show some respect.