Null (SQL)In SQL, null or NULL is a special marker used to indicate that a data value does not exist in the database. Introduced by the creator of the relational database model, E. F. Codd, SQL null serves to fulfill the requirement that all true relational database management systems (RDBMS) support a representation of "missing information and inapplicable information". Codd also introduced the use of the lowercase Greek omega (ω) symbol to represent null in database theory. In SQL, A null should not be confused with a value of 0. A null indicates a lack of a value, which is not the same as a zero value. For example, consider the question "How many books does Adam own?" The answer may be "zero" (we know that he owns none) or "null" (we do not know how many he owns). In a database table, the column reporting this answer would start with no value (marked by null), and it would not be updated with the value zero until it is ascertained that Adam owns no books. In SQL, null is a marker, not a value. This usage is quite different from most programming languages, where a null value of a reference means it is not pointing to any object. HistoryE. F. Codd mentioned nulls as a method of representing missing data in the relational model in a 1975 paper in the FDT Bulletin of ACM-SIGMOD. Codd's paper that is most commonly cited with the semantics of Null (as adopted in SQL) is his 1979 paper in the ACM Transactions on Database Systems, in which he also introduced his Relational Model/Tasmania, although much of the other proposals from the latter paper have remained obscure. Section 2.3 of his 1979 paper details the semantics of Null propagation in arithmetic operations as well as comparisons employing a ternary (three-valued) logic when comparing to nulls; it also details the treatment of Nulls on other set operations (the latter issue still controversial today). In database theory circles, the original proposal of Codd (1975, 1979) is now referred to as "Codd tables".[1] Codd later reinforced his requirement that all RDBMSs support Null to indicate missing data in a 1985 two-part article published in Computerworld magazine.[2][3] The 1986 SQL standard basically adopted Codd's proposal after an implementation prototype in IBM System R. Although Don Chamberlin recognized nulls (alongside duplicate rows) as one of the most controversial features of SQL, he defended the design of Nulls in SQL invoking the pragmatic arguments that it was the least expensive form of system support for missing information, saving the programmer from many duplicative application-level checks (see semipredicate problem) while at the same time providing the database designer with the option not to use Nulls if they so desire; for example, to avoid well-known anomalies (discussed in the semantics section of this article). Chamberlin also argued that besides providing some missing-value functionality, practical experience with Nulls also led to other language features that rely on Nulls, like certain grouping constructs and outer joins. Finally, he argued that in practice Nulls also end up being used as a quick way to patch an existing schema when it needs to evolve beyond its original intent, coding not for missing but rather for inapplicable information; for example, a database that quickly needs to support electric cars while having a miles-per-gallon column.[4] Codd indicated in his 1990 book The Relational Model for Database Management, Version 2 that the single Null mandated by the SQL standard was inadequate, and should be replaced by two separate Null-type markers to indicate why data is missing. In Codd's book, these two Null-type markers are referred to as 'A-Values' and 'I-Values', representing 'Missing But Applicable' and 'Missing But Inapplicable', respectively.[5] Codd's recommendation would have required SQL's logic system be expanded to accommodate a four-valued logic system. Because of this additional complexity, the idea of multiple Nulls with different definitions has not gained widespread acceptance in the database practitioners' domain. It remains an active field of research though, with numerous papers still being published. ChallengesNull has been the focus of controversy and a source of debate because of its associated three-valued logic (3VL), special requirements for its use in SQL joins, and the special handling required by aggregate functions and SQL grouping operators. Computer science professor Ron van der Meyden summarized the various issues as: "The inconsistencies in the SQL standard mean that it is not possible to ascribe any intuitive logical semantics to the treatment of nulls in SQL."[1] Although various proposals have been made for resolving these issues, the complexity of the alternatives has prevented their widespread adoption. Null propagationArithmetic operationsBecause Null is not a data value, but a marker for an absent value, using mathematical operators on Null gives an unknown result, which is represented by Null.[6] In the following example, multiplying 10 by Null results in Null: 10 * NULL -- Result is NULL
This can lead to unanticipated results. For instance, when an attempt is made to divide Null by zero, platforms may return Null instead of throwing an expected "data exception – division by zero".[6] Though this behavior is not defined by the ISO SQL standard many DBMS vendors treat this operation similarly. For instance, the Oracle, PostgreSQL, MySQL Server, and Microsoft SQL Server platforms all return a Null result for the following: NULL / 0
String concatenationString concatenation operations, which are common in SQL, also result in Null when one of the operands is Null.[7] The following example demonstrates the Null result returned by using Null with the SQL 'Fish ' || NULL || 'Chips' -- Result is NULL
This is not true for all database implementations. In an Oracle RDBMS, for example, NULL and the empty string are considered the same thing and therefore 'Fish ' || NULL || 'Chips' results in 'Fish Chips'.[8] Comparisons with NULL and the three-valued logic (3VL)Since Null is not a member of any data domain, it is not considered a "value", but rather a marker (or placeholder) indicating the undefined value. Because of this, comparisons with Null can never result in either True or False, but always in a third logical result, Unknown.[9] The logical result of the expression below, which compares the value 10 to Null, is Unknown: SELECT 10 = NULL -- Results in Unknown
However, certain operations on Null can return values if the absent value is not relevant to the outcome of the operation. Consider the following example: SELECT NULL OR TRUE -- Results in True
In this case, the fact that the value on the left of OR is unknowable is irrelevant, because the outcome of the OR operation would be True regardless of the value on the left. SQL implements three logical results, so SQL implementations must provide for a specialized three-valued logic (3VL). The rules governing SQL three-valued logic are shown in the tables below (p and q represent logical states)"[10] The truth tables SQL uses for AND, OR, and NOT correspond to a common fragment of the Kleene and Łukasiewicz three-valued logic (which differ in their definition of implication, however, SQL defines no such operation).[11]
Effect of Unknown in WHERE clausesSQL three-valued logic is encountered in Data Manipulation Language (DML) in comparison predicates of DML statements and queries. The SELECT *
FROM t
WHERE i = NULL;
The example query above logically always returns zero rows because the comparison of the i column with Null always returns Unknown, even for those rows where i is Null. The Unknown result causes the Null-specific and 3VL-specific comparison predicatesBasic SQL comparison operators always return Unknown when comparing anything with Null, so the SQL standard provides for two special Null-specific comparison predicates. The The SQL standard contains the optional feature F571 "Truth value tests" that introduces three additional logical unary operators (six in fact, if we count their negation, which is part of their syntax), also using postfix notation. They have the following truth tables:[13]
The F571 feature is orthogonal to the presence of the Boolean datatype in SQL (discussed later in this article) and, despite syntactic similarities, F571 does not introduce Boolean or three-valued literals in the language. The F571 feature was actually present in SQL92,[14] well before the Boolean datatype was introduced to the standard in 1999. The F571 feature is implemented by few systems however; PostgreSQL is one of those implementing it. The addition of IS UNKNOWN to the other operators of SQL's three-valued logic makes the SQL three-valued logic functionally complete,[15] meaning its logical operators can express (in combination) any conceivable three-valued logical function. On systems that don't support the F571 feature, it is possible to emulate IS UNKNOWN p by going over every argument that could make the expression p Unknown and test those arguments with IS NULL or other NULL-specific functions, although this may be more cumbersome. Law of the excluded fourth (in WHERE clauses)In SQL's three-valued logic the law of the excluded middle, p OR NOT p, no longer evaluates to true for all p. More precisely, in SQL's three-valued logic p OR NOT p is unknown precisely when p is unknown and true otherwise. Because direct comparisons with Null result in the unknown logical value, the following query SELECT * FROM stuff WHERE ( x = 10 ) OR NOT ( x = 10 );
is not equivalent in SQL with SELECT * FROM stuff;
if the column x contains any Nulls; in that case, the second query would return some rows the first one does not return, namely all those in which x is Null. In classical two-valued logic, the law of the excluded middle would allow the simplification of the WHERE clause predicate, in fact its elimination. Attempting to apply the law of the excluded middle to SQL's 3VL is effectively a false dichotomy. The second query is actually equivalent with: SELECT * FROM stuff;
-- is (because of 3VL) equivalent to:
SELECT * FROM stuff WHERE ( x = 10 ) OR NOT ( x = 10 ) OR x IS NULL;
Thus, to correctly simplify the first statement in SQL requires that we return all rows in which x is not null. SELECT * FROM stuff WHERE x IS NOT NULL;
In view of the above, observe that for SQL's WHERE clause a tautology similar to the law of excluded middle can be written. Assuming the IS UNKNOWN operator is present, p OR (NOT p) OR (p IS UNKNOWN) is true for every predicate p. Among logicians, this is called law of excluded fourth. There are some SQL expressions in which it is less obvious where the false dilemma occurs, for example: SELECT 'ok' WHERE 1 NOT IN (SELECT CAST (NULL AS INTEGER))
UNION
SELECT 'ok' WHERE 1 IN (SELECT CAST (NULL AS INTEGER));
produces no rows because SELECT 'ok' WHERE (1 IN (SELECT CAST (NULL AS INTEGER))) IS UNKNOWN;
Effect of Null and Unknown in other constructsJoinsJoins evaluate using the same comparison rules as for WHERE clauses. Therefore, care must be taken when using nullable columns in SQL join criteria. In particular a table containing any nulls is not equal with a natural self-join of itself, meaning that whereas is true for any relation R in relational algebra, a SQL self-join will exclude all rows having a Null anywhere.[16] An example of this behavior is given in the section analyzing the missing-value semantics of Nulls. The SQL (A = B) OR (A IS NULL AND B IS NULL)
CASE expressionsSQL provides two flavours of conditional expressions. One is called "simple CASE" and operates like a switch statement. The other is called a "searched CASE" in the standard, and operates like an if...elseif. The simple SELECT CASE i WHEN NULL THEN 'Is Null' -- This will never be returned
WHEN 0 THEN 'Is Zero' -- This will be returned when i = 0
WHEN 1 THEN 'Is One' -- This will be returned when i = 1
END
FROM t;
Because the expression On the other hand, a "searched" SELECT CASE WHEN i IS NULL THEN 'Null Result' -- This will be returned when i is NULL
WHEN i = 0 THEN 'Zero' -- This will be returned when i = 0
WHEN i = 1 THEN 'One' -- This will be returned when i = 1
END
FROM t;
In the searched Oracle's dialect of SQL provides a built-in function SELECT DECODE(i, NULL, 'Null Result', 0, 'Zero', 1, 'One') FROM t;
Finally, all these constructs return a NULL if no match is found; they have a default IF statements in procedural extensionsSQL/PSM (SQL Persistent Stored Modules) defines procedural extensions for SQL, such as the IF i = NULL THEN
SELECT 'Result is True'
ELSEIF NOT(i = NULL) THEN
SELECT 'Result is False'
ELSE
SELECT 'Result is Unknown';
The Analysis of SQL Null missing-value semanticsThe groundbreaking work of T. Imieliński and W. Lipski Jr. (1984)[17] provided a framework in which to evaluate the intended semantics of various proposals to implement missing-value semantics, that is referred to as Imieliński-Lipski Algebras. This section roughly follows chapter 19 of the "Alice" textbook.[18] A similar presentation appears in the review of Ron van der Meyden, §10.4.[1] In selections and projections: weak representationConstructs representing missing information, such as Codd tables, are actually intended to represent a set of relations, one for each possible instantiation of their parameters; in the case of Codd tables, this means replacement of Nulls with some concrete value. For example,
A construct (such as a Codd table) is said to be a strong representation system (of missing information) if any answer to a query made on the construct can be particularized to obtain an answer for any corresponding query on the relations it represents, which are seen as models of the construct. More precisely, if q is a query formula in the relational algebra (of "pure" relations) and if q is its lifting to a construct intended to represent missing information, a strong representation has the property that for any query q and (table) construct T, q lifts all the answers to the construct, i.e.: (The above has to hold for queries taking any number of tables as arguments, but the restriction to one table suffices for this discussion.) Clearly Codd tables do not have this strong property if selections and projections are considered as part of the query language. For example, all the answers to SELECT * FROM Emp WHERE Age = 22;
should include the possibility that a relation like EmpH22 may exist. However, Codd tables cannot represent the disjunction "result with possibly 0 or 1 rows". A device, mostly of theoretical interest, called conditional table (or c-table) can however represent such an answer:
where the condition column is interpreted as the row doesn't exist if the condition is false. It turns out that because the formulas in the condition column of a c-table can be arbitrary propositional logic formulas, an algorithm for the problem whether a c-table represents some concrete relation has a co-NP-complete complexity, thus is of little practical worth. A weaker notion of representation is therefore desirable. Imielinski and Lipski introduced the notion of weak representation, which essentially allows (lifted) queries over a construct to return a representation only for sure information, i.e. if it's valid for all "possible world" instantiations (models) of the construct. Concretely, a construct is a weak representation system if The right-hand side of the above equation is the sure information, i.e. information which can be certainly extracted from the database regardless of what values are used to replace Nulls in the database. In the example we considered above, it's easy to see that the intersection of all possible models (i.e. the sure information) of the query selecting If joins or unions are considered: not even weak representationConsider the following query over the same Codd table Emp from the previous section: SELECT Name FROM Emp WHERE Age = 22
UNION
SELECT Name FROM Emp WHERE Age <> 22;
Whatever concrete value one would choose for the
Thus when unions are added to the query language, Codd tables are not even a weak representation system of missing information, meaning that queries over them don't even report all sure information. It's important to note here that semantics of UNION on Nulls, which are discussed in a later section, did not even come into play in this query. The "forgetful" nature of the two sub-queries was all that it took to guarantee that some sure information went unreported when the above query was run on the Codd table Emp. For natural joins, the example needed to show that sure information may be unreported by some query is slightly more complicated. Consider the table
and the query SELECT F1, F3 FROM
(SELECT F1, F2 FROM J) AS F12
NATURAL JOIN
(SELECT F2, F3 FROM J) AS F23;
The intuition for what happens above is that the Codd tables representing the projections in the subqueries lose track of the fact that the Nulls in the columns F12.F2 and F23.F2 are actually copies of the originals in the table J. This observation suggests that a relatively simple improvement of Codd tables (which works correctly for this example) would be to use Skolem constants (meaning Skolem functions which are also constant functions), say ω12 and ω22 instead of a single NULL symbol. Such an approach, called v-tables or Naive tables, is computationally less expensive that the c-tables discussed above. However, it is still not a complete solution for incomplete information in the sense that v-tables are only a weak representation for queries not using any negations in selection (and not using any set difference either). The first example considered in this section is using a negative selection clause, Check constraints and foreign keysThe primary place in which SQL three-valued logic intersects with SQL Data Definition Language (DDL) is in the form of check constraints. A check constraint placed on a column operates under a slightly different set of rules than those for the DML CREATE TABLE t (
i INTEGER,
CONSTRAINT ck_i CHECK ( i < 0 AND i = 0 AND i > 0 ) );
Because of the change in designated values relative to the WHERE clause, from a logic perspective the law of excluded middle is a tautology for CHECK constraints, meaning In order to constrain a column to reject Nulls, the CREATE TABLE t ( i INTEGER NOT NULL );
By default check constraints against foreign keys succeed if any of the fields in such keys are Null. For example, the table CREATE TABLE Books
( title VARCHAR(100),
author_last VARCHAR(20),
author_first VARCHAR(20),
FOREIGN KEY (author_last, author_first)
REFERENCES Authors(last_name, first_name));
would allow insertion of rows where author_last or author_first are Outer joinsSQL outer joins, including left outer joins, right outer joins, and full outer joins, automatically produce Nulls as placeholders for missing values in related tables. For left outer joins, for instance, Nulls are produced in place of rows missing from the table appearing on the right-hand side of the The first table (Employee) contains employee ID numbers and names, while the second table (PhoneNumber) contains related employee ID numbers and phone numbers, as shown below.
The following sample SQL query performs a left outer join on these two tables. SELECT e.ID, e.LastName, e.FirstName, pn.Number
FROM Employee e
LEFT OUTER JOIN PhoneNumber pn
ON e.ID = pn.ID;
The result set generated by this query demonstrates how SQL uses Null as a placeholder for values missing from the right-hand (PhoneNumber) table, as shown below.
Aggregate functionsSQL defines aggregate functions to simplify server-side aggregate calculations on data. Except for the Note that the elimination of Null is not equivalent to replacing Null with zero. For example, in the following table,
Here The output of an aggregate function can also be Null. Here is an example: SELECT COUNT(*), MIN(e.Wage), MAX(e.Wage)
FROM Employee e
WHERE e.LastName LIKE '%Jones%';
This query will always output exactly one row, counting the number of employees whose last name contains "Jones", and giving the minimum and maximum wage found for those employees. However, what happens if none of the employees fit the given criteria? Calculating the minimum or maximum value of an empty set is impossible, so those results must be NULL, indicating there is no answer. This is not an Unknown value, it is a Null representing the absence of a value. The result would be:
When two nulls are equal: grouping, sorting, and some set operationsBecause SQL:2003 defines all Null markers as being unequal to one another, a special definition was required in order to group Nulls together when performing certain operations. SQL defines "any two values that are equal to one another, or any two Nulls", as "not distinct".[21] This definition of not distinct allows SQL to group and sort Nulls when the Other SQL operations, clauses, and keywords using the "not distinct" definition in their treatment of Nulls include:
The principle that Nulls are not equal to each other (but rather that the result is Unknown) is effectively violated in the SQL specification for the The SQL standard does not explicitly define a default sorting order for Nulls. Instead, on conforming systems, Nulls can be sorted before or after all data values by using the Effect on index operationSome SQL products do not index keys containing NULLs. For instance, PostgreSQL versions prior to 8.3 did not, with the documentation for a B-tree index stating that[22]
In cases where the index enforces uniqueness, NULLs are excluded from the index and uniqueness is not enforced between NULLs. Again, quoting from the PostgreSQL documentation:[23]
This is consistent with the SQL:2003-defined behavior of scalar Null comparisons. Another method of indexing Nulls involves handling them as not distinct in accordance with the SQL:2003-defined behavior. For example, Microsoft SQL Server documentation states the following:[24]
Both of these indexing strategies are consistent with the SQL:2003-defined behavior of Nulls. Because indexing methodologies are not explicitly defined by the SQL:2003 standard, indexing strategies for Nulls are left entirely to the vendors to design and implement. Null-handling functionsSQL defines two functions to explicitly handle Nulls: NULLIFThe NULLIF(value1, value2)
Thus, CASE WHEN value1 = value2 THEN NULL ELSE value1 END
COALESCEThe COALESCE(value1, value2, value3, ...)
CASE WHEN value1 IS NOT NULL THEN value1
WHEN value2 IS NOT NULL THEN value2
WHEN value3 IS NOT NULL THEN value3
...
END
Some SQL DBMSs implement vendor-specific functions similar to NVLThe Oracle A COALESCE ( val1, ... , val{n} )
turns into: NVL( val1 , NVL( val2 , NVL( val3 , … , NVL ( val{n-1} , val{n} ) … )))
A use case of this function is to replace in an expression a NULL by a value like in There is, however, one notable exception. In most implementations, Data typing of Null and UnknownThe Conversion from the CAST (NULL AS INTEGER)
represents an absent value of type INTEGER. The actual typing of Unknown (distinct or not from NULL itself) varies between SQL implementations. For example, the following SELECT 'ok' WHERE (NULL <> 1) IS NULL;
parses and executes successfully in some environments (e.g. SQLite or PostgreSQL) which unify a NULL Boolean with Unknown but fails to parse in others (e.g. in SQL Server Compact). MySQL behaves similarly to PostgreSQL in this regard (with the minor exception that MySQL regards TRUE and FALSE as no different from the ordinary integers 1 and 0). PostgreSQL additionally implements a BOOLEAN data typeThe ISO SQL:1999 standard introduced the BOOLEAN data type to SQL, however it's still just an optional, non-core feature, coded T031.[27] When restricted by a The Boolean type has been subject of criticism, particularly because of the mandated behavior of the UNKNOWN literal, which is never equal to itself because of the identification with NULL.[30] As discussed above, in the PostgreSQL implementation of SQL, Null is used to represent all UNKNOWN results, including the UNKNOWN BOOLEAN. PostgreSQL does not implement the UNKNOWN literal (although it does implement the IS UNKNOWN operator, which is an orthogonal feature.) Most other major vendors do not support the Boolean type (as defined in T031) as of 2012.[31] The procedural part of Oracle's PL/SQL however supports BOOLEAN variables; these can also be assigned NULL and the value is considered the same as UNKNOWN.[32] ControversyCommon mistakesMisunderstanding of how Null works is the cause of a great number of errors in SQL code, both in ISO standard SQL statements and in the specific SQL dialects supported by real-world database management systems. These mistakes are usually the result of confusion between Null and either 0 (zero) or an empty string (a string value with a length of zero, represented in SQL as A classic error is the attempt to use the equals operator SELECT *
FROM sometable
WHERE num = NULL; -- Should be "WHERE num IS NULL"
In a related, but more subtle example, a SELECT *
FROM sometable
WHERE num <> 1; -- Rows where num is NULL will not be returned,
-- contrary to many users' expectations.
These confusions arise because the Law of Identity is restricted in SQL's logic. When dealing with equality comparisons using the Similarly, Nulls are often confused with empty strings. Consider the SELECT *
FROM sometable
WHERE LENGTH(string) < 20; -- Rows where string is NULL will not be returned.
This is complicated by the fact that in some database interface programs (or even database implementations like Oracle's), NULL is reported as an empty string, and empty strings may be incorrectly stored as NULL. CriticismsThe ISO SQL implementation of Null is the subject of criticism, debate and calls for change. In The Relational Model for Database Management: Version 2, Codd suggested that the SQL implementation of Null was flawed and should be replaced by two distinct Null-type markers. The markers he proposed were to stand for "Missing but Applicable" and "Missing but Inapplicable", known as A-values and I-values, respectively. Codd's recommendation, if accepted, would have required the implementation of a four-valued logic in SQL.[5] Others have suggested adding additional Null-type markers to Codd's recommendation to indicate even more reasons that a data value might be "Missing", increasing the complexity of SQL's logic system. At various times, proposals have also been put forth to implement multiple user-defined Null markers in SQL. Because of the complexity of the Null-handling and logic systems required to support multiple Null markers, none of these proposals have gained widespread acceptance. Chris Date and Hugh Darwen, authors of The Third Manifesto, have suggested that the SQL Null implementation is inherently flawed and should be eliminated altogether,[34] pointing to inconsistencies and flaws in the implementation of SQL Null-handling (particularly in aggregate functions) as proof that the entire concept of Null is flawed and should be removed from the relational model.[35] Others, like author Fabian Pascal, have stated a belief that "how the function calculation should treat missing values is not governed by the relational model."[citation needed] Closed-world assumptionAnother point of conflict concerning Nulls is that they violate the closed-world assumption model of relational databases by introducing an open-world assumption into it.[36] The closed world assumption, as it pertains to databases, states that "Everything stated by the database, either explicitly or implicitly, is true; everything else is false."[37] This view assumes that the knowledge of the world stored within a database is complete. Nulls, however, operate under the open world assumption, in which some items stored in the database are considered unknown, making the database's stored knowledge of the world incomplete. See also
References
Further reading
External links |