I’ve just ported LIVR to Perl6. It was really fun to code in Perl6. Moreover, LIVR’s test suite allowed me to find a bug in Perl6 Email::Valid, and another one in Rakudo itself. It was even more fun that you not just implemented module but helped other developers to do some testing :)
What is LIVR? LIVR stands for “Language Independent Validation Rules”. So, it is like “Mustache” but in the world of validation. So, LIVR consists of the following parts:
-
Universal test suite, that is used for checking that the implementation works properly.
There is LIVR for:
-
Perl 5 (LIVR 2.0) available at CPAN, maintainer @koorchik
-
Perl 6 (LIVR 2.0) available at CPAN, maintainer @koorchik
-
JavaScript (LIVR 2.0) available at npm, maintainer @koorchik
-
PHP (LIVR 2.0) available at packagist, maintainer @WebbyLab
-
Python (LIVR 2.0) available at pypi, maintainer @asholok
-
OLIFER Erlang (LIVR 2.0), maintainer @Prots
-
LIVER Erlang (LIVR 2.0), maintainer @erlangbureau
-
Java (LIVR 2.0), maintainer @vlbaluk
-
Ruby (LIVR 0.4, previous version) at rubygems, maintainer @maktwin
I will give you a short intro about LIVR here but for details, I strongly recommend to read this post “LIVR – Data Validation Without Any Issues”.
LIVR Intro
Data validation is a very common task. I am sure that every developer faces it again and again. Especially, it is important when you develop a Web application. It is a common rule – never trust user’s input. It seems that if the task is so common, there should be tons of libraries. Yes, it is but it is very difficult to find one that is ideal. Some of the libraries are doing too many things (like HTML form generation etc), other libraries are difficult to extend, some does not have hierarchical data support etc.
Moreover, if you are a web developer, you could need the same validation on the server and on the client.
In WebbyLab, mainly we use 3 programming languages – Perl, JavaScript, PHP. So, for us, it was ideal to reuse similar validation approach across languages.
Therefore, it was decided to create a universal validator that could work across different languages.
Validator Requirements
After trying tons of validation libraries, we had some vision in our heads about the issues we want to solve. Here are the requirements for the validator:
-
Rules are declarative and language independent. So, rules for validation is just a data structure, not method calls etc. You can transform it, change it as you do this with any other data structure.
-
Any number of rules for each field.
-
The validator should return together errors for all fields. For example, we want to highlight all errors in a form.
-
Cut out all fields that do not have validation rules described. (otherwise, you cannot rely on your validation, someday you will have a security issue if the validator will not meet this property).
-
Possibility to validate complex hierarchical structures. Especially useful for JSON APIs.
-
Easy to describe and understand validation.
-
Returns understandable error codes (neither error messages nor numeric codes)
-
Easy to implement own rules (usually you will have several in every project)
-
Rules should be able to change results output (“trim”, “nested_object”, for example)
-
Multipurpose (user input validation, configs validation etc)
-
Unicode support.
LIVR Specification
Since the task was set to create a validator independent of a programming language (some kind of a mustache/handlebars stuff) but within the data validation sphere, we started with the composition of specifications.
The specifications’ objectives are:
-
To standardize the data description format.
-
To describe a minimal set of the validation rules that must be supported by every implementation.
-
To standardize error codes.
-
To be a single basic documentation for all the implementations.
-
To feature a set of testing data that allows checking if the implementation fits the specifications.
-
The basic idea was that the description of the validation rules must look like a data scheme and be as similar to data as possible, but with rules instead of values.
The specification is available here http://livr-spec.org/.
This is the basic intro. More details are in the post I’ve mentioned above.
LIVR and Perl6
Let’s have some fun and play with a code. I will go through several examples, and will provide some internal details after each example. The source code of all examples is available on GitHub
At first, install LIVR module for Perl6 from CPAN
| zef install LIVR |
Example 1: registration data validation
| use LIVR; | |
| # Automatically trim all values before validation | |
| LIVR::Validator.default-auto-trim(True); | |
| my $validator = LIVR::Validator.new(livr-rules => { | |
| name => 'required', | |
| email => [ 'required', 'email' ], | |
| gender => { one_of => ['male', 'female'] }, | |
| phone => { max_length => 10 }, | |
| password => [ 'required', {min_length => 10} ], | |
| password2 => { equal_to_field => 'password' } | |
| }); | |
| my $user-data = { | |
| name => 'Viktor', | |
| email => 'viktor@mail.com', | |
| gender => 'male', | |
| password => 'mypassword123', | |
| password2 => 'mypassword123' | |
| } | |
| if my $valid-data = $validator.validate($user-data) { | |
| # $valid-data is clean and does contain only fields | |
| # which have validation and have passed it | |
| $valid-data.say; | |
| } else { | |
| my $errors = $validator.errors(); | |
| $errors.say; | |
| } |
So, how to understand the rules?
The idea is very simple. Each rule is a hash. key – name of the validation rules. value – an array of arguments.
For example:
| { | |
| name => { required => [] }, | |
| phone => { max_length => [10] } | |
| } |
but if there is only one argument, you can use a shorter form:
| { | |
| phone => { max_length => 10 } | |
| } |
if there are no arguments, you can just pass the name of the rule as string
| { | |
| name => 'required' | |
| } |
you can pass a list of rules for a field in an array:
| { | |
| name => [ 'required', { max_length => 10 } ] | |
| } |
In this case, rules will be applied one after another. So, in this example, at first, the “required” rule will be applied and “max_length” after that and only if the “required” passed successfully.
Here is the details in LIVR spec.
You can find the list of standard rules here.
Example 2: validation of hierarchical data structure
| use LIVR; | |
| my $validator = LIVR::Validator.new(livr-rules => { | |
| name => 'required', | |
| phone => {max_length => 10}, | |
| address => {'nested_object' => { | |
| city => 'required', | |
| zip => ['required', 'positive_integer'] | |
| }} | |
| }); | |
| my $user-data = { | |
| name => "Michael", | |
| phone => "0441234567", | |
| address => { | |
| city => "Kiev", | |
| zip => "30552" | |
| } | |
| } | |
| if my $valid-data = $validator.validate($user-data) { | |
| # $valid-data is clean and does contain only fields | |
| # which have validation and have passed it | |
| $valid-data.say; | |
| } else { | |
| my $errors = $validator.errors(); | |
| $errors.say; | |
| } |
What is interesting in this example?
-
The schema (validation rules) shape looks very similar to the data shape. It is much easier to read than JSON Schema, for example.
-
It seems that “nested_object” is a special syntax but it is not. The validator does not make any difference between “required”, “nested_object” “max_length’. So, the core is very tiny and you can introduce a new feature easily with custom rules.
-
Often you want to reuse complex validation rules like ‘address’ and it can be done with aliasing.
-
You will receive a hierarchical error message. For example, if you will miss city and name, the error object will look
{name => 'REQUIRED', address => {city => 'REQUIRED'} }.
Aliases
| use LIVR; | |
| LIVR::Validator.register-aliased-default-rule({ | |
| name => 'short_address', # names of the rule | |
| rules => {'nested_object' => { | |
| city => 'required', | |
| zip => ['required', 'positive_integer'] | |
| }}, | |
| error => 'WRONG_ADDRESS' # custom error (optional) | |
| }); | |
| my $validator = LIVR::Validator.new(livr-rules => { | |
| name => 'required', | |
| phone => {max_length => 10}, | |
| address => 'short_address' | |
| }); | |
| my $user-data = { | |
| name => "Michael", | |
| phone => "0441234567", | |
| address => { | |
| city => "Kiev", | |
| zip => "30552" | |
| } | |
| } | |
| if my $valid-data = $validator.validate($user-data) { | |
| # $valid-data is clean and does contain only fields | |
| # which have validation and have passed it | |
| $valid-data.say; | |
| } else { | |
| my $errors = $validator.errors(); | |
| $errors.say; | |
| } |
If you want, you can register aliases only for your validator instance:
| use LIVR; | |
| my $validator = LIVR::Validator.new(livr-rules => { | |
| password => ['required', 'strong_password'] | |
| }); | |
| $validator.register-aliased-rule({ | |
| name => 'strong_password', | |
| rules => {min_length => 6}, | |
| error => 'WEAK_PASSWORD' | |
| }); |
Example 3: data modification, pipelining
There are rules that can do data modification. Here is the list of them:
-
trim
-
to_lc
-
to_uc
-
remove
-
leave_only
-
default
You can read details here.
With such approach, you can create some sort of pipe.
| use LIVR; | |
| my $validator = LIVR::Validator.new(livr-rules => { | |
| email => [ 'trim', 'required', 'email', 'to_lc' ] | |
| }); | |
| my $input-data = { email => ' EMail@Gmail.COM ' }; | |
| my $output-data = $validator.validate($input-data); | |
| $output-data.say; |
What is important here?
-
As I mentioned before, for the validator there is no difference between any of the rules. It treats “trim”, “default”, “required”, “nested_object” the same way.
-
Rules are applied one after another. The output of a rule will be passed to the input of the next rule. It is like a bash pipe
echo ' EMail@Gmail.COM ' | trim | required | email | to_lc -
$input-data will be NEVER changed. $output-data is data you use after the validation.
Example 4: custom rules
You can use aliases as custom rules but sometimes it is not enough. It is absolutely fine to write an own custom rule. You can do almost everything with custom rules.
Usually, we have 1-5 custom rules almost in every our project. Moreover, you can organize custom rules as a separate reusable module (even upload it to CPAN).
So, how to write a custom rule for LIVR?
Here is the example of ‘strong_password’:
| use LIVR; | |
| my $validator = LIVR::Validator.new(livr-rules => { | |
| password => ['required', 'strong_password'] | |
| }); | |
| $validator.register-rules( 'strong_password' => sub (@rule-args, %builders) { | |
| # %builders - are rules from original validator | |
| # to allow you create new validator with all supported rules | |
| # my $validator = LIVR::Validator.new(livr-rules => $livr).register-rules(%builders).prepare(); | |
| # See "nested_object" rule implementation for example | |
| # https://github.com/koorchik/perl6-livr/blob/master/lib/LIVR/Rules/Meta.pm6#L5 | |
| # Return closure that will take value and return error | |
| return sub ($value, $all-values, $output is rw) { | |
| # We already have "required" rule to check that the value is present | |
| return if LIVR::Utils::is-no-value($value); # so we skip empty values | |
| # Return value is a string | |
| return 'FORMAT_ERROR' if $value !~~ Str && $value !~~ Numeric; | |
| # Return error in case of failed validation | |
| return 'WEAK_PASSWORD' if $value.chars < 6; | |
| # Change output value. We want always return value be a string | |
| $output = $value.Str; | |
| return; | |
| }; | |
| }); |
Look at existing rules implementation for more examples:
Example 5: Web application
LIVR works great for REST APIs. Usually, a lot of REST APIs have a problem with returning understandable errors. If a user of your API will receive HTTP error 500, it will not help him. Much better when he will get errors like
| { | |
| "name": "REQUIRED", | |
| "phone": "TOO_LONG", | |
| "address": { | |
| "city": "REQUIRED", | |
| "zip": "NOT_POSITIVE_INTEGER" | |
| } | |
| } |
than just “Server error”.
So, let try to do a small web service with 2 endpoints:
-
GET /notes -> get list of notes
-
POST /notes -> create a note
You will need to install Bailador for it:
| zef install Bailador |
Let’s create some services. I prefer “Command” pattern for the services with template method “run”.
We will have 2 services:
-
Service::Notes::Create
-
Service::Notes::List
Service usage example:
| my %CONTEXT = (storage => my @STORAGE); | |
| my %note = title => 'Note1', text => 'Note text'; | |
| my $new-note = Service::Notes::Create.new( | |
| context => %CONTEXT | |
| ).run(%note); | |
| my $list = Service::Notes::Create.new( | |
| context => %CONTEXT | |
| ).run({}); |
With context you can inject any dependencies. “run” method accepts data passed by user.
Here is how the source code of the service for notes creation looks like:
| use Service::Base; | |
| my $LAST_ID = 0; | |
| class Service::Notes::Create is Service::Base { | |
| has %.validation-rules = ( | |
| title => ['required', {max_length => 20} ], | |
| text => ['required', {max_length => 255} ] | |
| ); | |
| method execute(%note) { | |
| %note<id> = $LAST_ID++; | |
| $.context<storage>.push(%note); | |
| return %note; | |
| } | |
| } |
and the Service::Base class:
| use LIVR; | |
| LIVR::Validator.default-auto-trim(True); | |
| class Service::Base { | |
| has $.context = {}; | |
| method run(%params) { | |
| my %clean-data = self!validate(%params); | |
| return self.execute(%params); | |
| } | |
| method !validate($params) { | |
| return $params unless %.validation-rules.elems; | |
| my $validator = LIVR::Validator.new( | |
| livr-rules => %.validation-rules | |
| ); | |
| if my $valid-data = $validator.validate($params) { | |
| return $valid-data; | |
| } else { | |
| die $validator.errors(); | |
| } | |
| } | |
| } |
“run” method guarantees that all procedures are kept:
-
Data was validated.
-
“execute” will be called only after validation.
-
“execute” will receive only clean data.
-
Throws an exception in case of validation errors.
-
Can check permissions before calling “execute”.
-
Can do extra work like caching validator objects, etc.
Here is the full working example.
Run the app:
| perl6 app.pl6 |
Create a note:
| curl -H "Content-Type: application/json" -X POST -d '{"title":"New Note","text":"Some text here"}' http://localhost:3000/notes |
Check validation:
| curl -H "Content-Type: application/json" -X POST -d '{"title":"","text":""}' http://localhost:3000/notes |
Get the list of notes:
| curl http://localhost:3000/notes |
LIVR links
-
You can play online with LIVR Playground
-
You can play online with LIVR Multi-Language Playground
I hope you will like the LIVR. I will appreciate any feedback.
Raku Advent Calendar (2009-2019) 