ETL and Data Integration tools - Modern ETL software for everyone

Home Products Advanced ETL Processor EntData Validation

Data Validation

Regular Expressions

Checking quality of the data

Advanced ETL Processor is able to check any data, for exampe date formats, post codes, phone numbers etc. It is also possible to write your own Regular expressions

Checking File format

Checking source data

What are the regular expressions?

Regular expression is a string that is used to describe or match a set of strings, according to certain syntax rules. Regular expressions are used by many text editors, utilities, and programming languages to search and manipulate text based on patterns. For example, Perl and Tcl have a powerful regular expression engine built directly into their syntax. Several utilities provided by Unix distributions—including the editor ed and the filter grep—were the first to popularize the concept of regular expressions.
"Regular expression" is often shortened to regex or regexp (singular), or regexes, regexps, or regexen (plural). Some authors distinguish between regular expression and abbreviated forms such as regex, restricting the former to true regular expressions, which describe regular languages, while using the latter for any regular expression-like pattern, including those that describe languages that are not regular. As only some authors observe this distinction, it is not safe to rely upon it. As an example of the syntax, the regular expression \bex can be used to search for all instances of the string "ex" that occur at word boundaries (signified by the \b). Thus in the string, "Texts for experts," \bex matches the "ex" in "experts," but not in "Texts" (because the "ex" occurs inside the word there and not immediately after a word boundary).

Source: wikipedia.

Below is a list of Date Validation Functions supported by Advanced ETL Processor

Image	Data Validation Function	Description
	Is Date	Checks if the data is a Date
	Is Date Between	Checks if the data is a Date between two values
	Is Date Less Than	Checks if the data is a Date less than value
	Is Date More Than	Checks if the data is a Date more than value
	Is Monday, Is Tuesday, Is Wednesday, Is Thursday, Is Friday, Is Saturday, Is Sunday, Is Today, Is Yesterday, Is Tomorrow, Is Weekend, Is Weekday	Checks if the data is a Date value which belongs to specific day of week
	Is January, Is February, Is March, Is April, Is May, Is June, Is July, Is August, Is September, Is October, Is November, Is December, Is Current Month, Is Last Month, Is Next Month	Checks if the data is a Date value which belongs to specific month of a year
	Is 1st Quarter, Is 2nd Quarter, Is 3rd Quarter, Is 4th Quarter, Is Current Quarter,Is Last Quarter, Is Next Quarter	Checks if the data is a Date value which belongs to specific quarter of a year
	Is Leap Year, Is Current Year, Is Last Year, Is Next Year	Checks if the data is a Date value which belongs to specific year
	Is Current Week, Is Last Week, Is Next Week	Checks if the data is a Date value which belongs to specific week
	Is Within Past Minutes, Is Within Past Hours, Is Within Past Days, Is Within Past Weeks, Is Within Past Months	Checks if the data is a Date value within in past minutes etc.

Validating Numbers

Below is a list of Number Validation Functions supported by Advanced ETL Processor

Image	Data Validation Function	Description
	Is Number	Checks if the data is a Number
	Is Integer	Checks if the data is an Integer
	Is Positive	Checks if the data is a Positive Number
	Is Negative	Checks if the data is a Negative Number
	Is Number Between	Checks if the data is a number between two values
	Is Number Less Than	Checks if the data is a number less than value
	Is Number More Than	Checks if the data is a number more than value

Validating Strings

Below is a list of String Validation Functions supported by Advanced ETL Processor

Image	Data Validation Function	Description
	Is Null	Checks if the data is equal to null.
	Is Empty String	Checks if the data is Empty String. (Length of the string is zero).
	Is Alpha	Checks if the data contains only Alpha characters.
	Is Alpha Numeric	Checks if the data contains only Alpha Numeric characters.
	Is Hex	Checks if the data contains valid HEX string
	Is Equal To	Checks if the data is equal to supplied value
	Is Starts With	Checks if the data starts with supplied value
	Is Ends With	Checks if the data ends with supplied value
	Contains	Checks if the data contains supplied value
	In List	Checks if the data exists within supplied list values

Time Validation

Below is list of Time Validation Functions supported by Advanced ETL Processor

Image	Data Validation Function	Description
	Is Time	Checks if the data is a time value
	Is Time Between	Checks if the data is a time between two values
	Is Time Less Than	Checks if the data is a time less than value
	Is Time More Than	Checks if the data is a time more than value
	Is Second	Checks if data is a number between 0 and 59
	Is Minute	Checks if data is a number between 0 and 59
	Is Hour 24	Checks if data is a number between 0 and 23
	Is Hour 12	Checks if data is a number between 0 and 12
	Is PM	Checks if the data is a PM Date
	Is AM	Checks if the data is a AM Date