en_dmitriid


Tigers, and lions, and bears, oh my!


Shame on me
happy
dmitriid
Shame on me. I had a mistake in my previous pos. Thanks to xaprb for pointing this out.

I misplaced quote marks in my second query.

Instead of

date BETWEEN '2008-01-01 12:00:00.000' AND '2008-01-14 12:00:00.000' AND accomodation=1


I wrote this:

date BETWEEN '2008-01-01 12:00:00.000' AND '2008-01-14 12:00:00.000 AND accomodation=1'


Note the last quote mark after accomodation=1

:)
Tags: ,

Those evening queries...
happy
dmitriid
I'm dealing with databases right now while keeping an eye on «High performance MySQL». Finally decided to try something out. Here we go...

We have a table split into partitions be week:


CREATE TABLE `price_detail` (
`accomodation` int(11) NOT NULL,
`date` datetime NOT NULL,
`price` int(11) NOT NULL,
`room_id` int(11) NOT NULL,
KEY `idx_price_date` (`date`) USING BTREE,
KEY `idx_price` (`price`) USING BTREE,
KEY `idx_price_accomodation` (`accomodation`) USING BTREE
)
PARTITION BY RANGE (TO_DAYS(`date`)) (
PARTITION p20080101 VALUES LESS THAN (733407),
PARTITION p20080108 VALUES LESS THAN (733414),
/* and so on — an entire years worth */


Yep... We have indexes as well...

We have 26 million rows per week in the worst case. Needless to say that we have the worst case for testing purposes .

Let's see how a simple query will work on such a database:

Query I:

explain partitions 
  select 
    accomodation 
  from 
    price_detail 
  where 
    accomodation=1 and date BETWEEN '2008-01-01 12:00:00.000' AND '2008-01-14 12:00:00.000'\G;

*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: price_detail
   partitions: p20080108,p20080115
         type: ref
possible_keys: idx_price_date,idx_price_accomodation
          key: idx_price_accomodation
      key_len: 4
          ref: const
         rows: 40292
        Extra: Using where


Query II:


explain partitions 
  select 
    accomodation 
  from 
    price_detail 
  where 
    date BETWEEN '2008-01-01 12:00:00.000' AND '2008-01-14 12:00:00.000 AND accomodation=1'\G;

*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: price_detail
   partitions: p20080108,p20080115,p20080122,p20080129,p20080205,p20080212,p20080219,
                     p20080226,p20080304,p20080311,p20080318,p20080325,p20080401,p20080408,
                     p20080415,p20080422,p20080429,p20080506,p20080513,p20080520,p20080527,
                     p20080603,p20080610,p20080617,p20080624,p20080701,p20080708,p20080715,
                     p20080722,p20080729,p20080805,p20080812,p20080819,p20080826,p20080902,
                     p20080909,p20080916,p20080923,p20080930,p20081007,p20081014,p20081021,
                     p20081028,p20081104,p20081111,p20081118,p20081125,p20081202,p20081209,
                     p20081216,p20081223,p20081230,p20090106,p20090113,p20090120,p20090127
         type: ALL
possible_keys: idx_price_date
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 274845000
        Extra: Using where


What a nasty surprise!

I'm off learning about covering indices...
Tags: ,

For whom the headhunt tolls...
happy
dmitriid
We're hiring now.

The first line in job description reads: Good or excellent knowledge of PHP

Here's what we get: "I've just started learning РНР...»

Perhaps, we should've stated our requirements more clearly... Somehow... :)
Tags:

A hundred wise men
happy
dmitriid
via http://andrzejn.livejournal.com/1152676.html

- A hundred wise men know that something is impossible. Then a fool comes along...

- And he makes a discovery?

- No. However, the wise men remember him with tenderness for long afterwards.

Musings
happy
dmitriid
As I promised here are my thoughts:

1. First of all, pattern matching lets you describe what yo eed with little or no effort. For example, consider these rules:

{field}
{field, Func}
{field, {'=', field2}}


Frankly, I have a very vague idea how these could be matched using ifs only. I guess you would create a Rule class and inherit a slew of classes from it. Something like RuleField, RuleFunc, RuleOperator and so on. If you use pattern matching, however, the rules are easily parsed::

%%{field}
validate_rule({FieldName}) ->
    ok.

%%{field, Func}
validate_rule({FieldName, Func}) when is_function(Func) ->
    ok.

%%{field, {'=', field2}}
validate_rule({FieldName, {Operator, FieldName2}}) ->
    ok.


Such ease, however, may bring (and it does bring me) to my second thought:

2. WTF-ish code. The thought that you can easily slice through complex constructs makes you write before you think.

In my case I knew what I wanted to pass to the function, but I had no idea what I expected the function to return. Ok, I've parsed the rules, what now?

As a result, the first version of the function would return a deeply nested list of lists that looked something like this:


[[Field, [Error1, Error2]], [Field2, [Error3, Error4]]]


It took me two additional refactorings to make it return a properproplist.

This same "wow, look at how I handle things!" approach resulted in an ugly preprocessing of results before I return these to the user:

lists:filter(
    fun(Elem) -> 
        case 
            Elem of {} -> false; 
            {_, []} -> false;
            _ -> true 
        end 
    end, 
    lists:flatten(validate1(A, ValidationRules, [])))


Yup. Get rid of those unwanted elements before the user sees them. Are you scared? I am. I m saddened as well :(


Here's where my train of thoughts stops...
Tags:

Validation for erlyweb
happy
dmitriid
I've written a validation function for Erlyweb. First I'll tell you how it works and then, perhaps, more thoughts on its internals.

Create a form with four fields:
- login
- password
- pasword_repeat
- email

This is a pretty standard registration form. Naturally we'd have to validate input coming from this form:
- login has to be 4-16 symbols in length
- password has to be 4-16 symbols in length
- password has to be the same as password_repeat
- email has to be a valid email address

Erlyweb's validation functions cant cope with this. My function can :)

Suppose you have a function called process_signup, which accepts the yaws_arg record. Then the validation will look like so:

process_signup(A) ->
	F = fun(A, Field) ->
		{ok, Val} = yaws_api:postvar(A, Field),
		L = string:len(Val),
		if
			L < 4 orelse L > 16 ->
				{Field, length};
			true ->
				{}
			end
	end,
	EmailCheck = fun(Args, Field2) ->
		{ok, Email} = yaws_api:postvar(Args, Field2),
		Match = regexp:match(Email, "^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]+"),
		Match /= nomatch
	end,

	%% magic is here:
	buktu_form:validate(A, [
		{login, F},
		{email, EmailCheck},
		{password, [{'=', password_repeat}, F]}
	]).


If we don't input any field at all, we'll get back the following list:
[{login,invalid_field},
 {email,invalid_field},
 {password,[{invalid_fields,[password,password_repeat]},
            invalid_field]}]


If we input values that don't match our criteria, we'll get:
[{login,length},
 {email,invalid_value},
 {password,[{not_equal,password_repeat},
            length]}]


  • The callback function that you can pass can return the following:

    • a tuple {FieldName, Error}

    • true if the field is validated and false otherwise (then the validation function will return {FieldName, invalid_value})

    • any value Value which will be transformed into {FieldName, Value}


  • if any of the fields in the rule don't exist or are empty, for each such rule the validation function will return:

    • invalid_field if the field is compared against a value

    • {invalid_fields, [field1, field2]} if the field is compared against another field


  • If you need to bind several rules to a field, pass a list of rules. If you just need to check if a field exists, pass in a tuple containing the field's name:

    • Does the field exist?
      {field_name}

    • Compare a field field_name to field field2_name (you can use '=', '/=', '<', '=<', '>', '>=' )
      {field_name, {'=', field2_name}}

    • Compare a field to any value:
      {field_name, {'=', Value}}

    • Use a callback function (function/2, first parameter is yaws_arg, the second is the field's name). The function can be a lambda or any function of any module in the form of module:function/2 or {module, function}
      {field_name, F}

    • Use a callback function with an additional value (function/3,first parameter is yaws_arg, the second is the field's name, the third is the value). The function can be a lambda or any function of any module in the form of module:function/3 or {module, function}
      {field_name, {F, Value}}


  • The validation function returns a proplist:
    [{FieldName(), Errors()}]

    where
    FieldName() = atom()
    Errors() = Error() | [Error()]
    Error() = user_defined_values | absent | invalid_value | 
              invalid_field | {invalid_fields, [FieldName(), FieldName()]} |
              ComparisonError()
    ComparisonError() = {not_equal, value_or_field} | {equal, value_or_field} |
                        {not_greater, value_or_field} | {not_less, value_or_field} |
                        {not_greater_or_equal, value_or_field} | {not_less_or_equal, value_or_field} |
    

Apple...
happy
dmitriid



there it is.... quite unusual, but there is no way back...
Tags:

Somebody stop me
happy
dmitriid
Or shoot me before I suffer too long :)

I've managed to get one of the oldest Firefox bugs going: https://bugzilla.mozilla.org/show_bug.cgi?id=69230 (see comment 86). I have know idea what to do with it now :) And there's the possibility that I will have to compile Mozilla from the source...

"Tell my wife and children that I love them..." :))

Erlang Bit Syntax
happy
dmitriid
I've decided to see whether all is as well with Erlang's bit syntax as advertised..

A friend of mine has been struggling with SOny's OMA format which is being used in Sony's players. The only feature this format has that distinguishes it from "normal" formats is its lack of documentation and whatever documentation there is differs significantly from whatever actual files have to offer :)Read more...Collapse )
Tags:

Programming Erlang
happy
dmitriid
It has finally arrived!

Programming Erlang Programming Erlang Programming Erlang

It arrived at Moldova on Juky 21st. It got out of customs on July 25th. And it is only today that I finally noticed the invitation to pick it up at my local post office. :)

Hurrah!

Now if I only could find time to study it...
Tags:

?

Log in