Add support for extensible records

Samuel Mimram added a comment - 29/Oct/11 4:35 PM For the record, I started to implement this. Commits soon ahead...

Samuel Mimram added a comment - 29/Oct/11 5:38 PM Work started on branch LS-592 . Please see the comments of commit 8a78c5b018bc review the patch!

Samuel Mimram added a comment - 30/Oct/11 3:20 PM Ok, so records seem to work right now (needs a bit more testing to be sure though). However, before thinking of merging I would like to have the "." notation for accessing fields of records (right now, it is r$field). This means that we should use the dot as a record access everywhere, e.g. in "list.hd", and not handle it as a character in the function name. I have added preliminary functions in order for the migration to be easy. Right now, you can type def math$add (x, y) = x + y end def math$inv (x) = 1 / x end and if you run this with liquidsoap -i, you will get math : [add:((?A,?A)->?A)] where ?A is a number type math : [inv:((int)->int);add:((?A,?A)->?A)] where ?A is a number type i.e. math is a record to which are progressively added members (actually new records are defined, each one masking the preceeding). It would be easy to extend the mechanism for builtins, but we have a problem: currently (if I understand well) the builtins come with type schemes (and not types). This means that we have to handle properly type schemes with records. I am currently thinking about this...

Samuel Mimram added a comment - 30/Oct/11 4:25 PM This is now "done" :) I am waiting for testing and comments now!

Samuel Mimram added a comment - 30/Oct/11 4:32 PM Actually, I just found out that there are shift/reduce conflicts in my latest commit, but I'm too lazy to correct them right now (they might even force us to rethink the grammar, the VARLPAR hack in particular...)

David Baelde added a comment - 30/Oct/11 6:27 PM Good point about nested polymorphism. This isn't a big problem in theory, but in practice, it requires to change the type representation.

Samuel Mimram added a comment - 30/Oct/11 6:30 PM Yep we need nested polymorphism, and I don't know how to properly do that (I don't fully understand the type schemes representation yet). In the current implementation, I have a type scheme at record level, not field level.

Samuel Mimram added a comment - 31/Oct/11 8:15 AM Apart from nested polymorphism, another point we should think about. Right now, we use a lot of things like: %ifdef... video.text = video.text.gd %endif %ifdef... video.text = video.text.sdl %endif which won't work anymore because the first definition will mask the record video.text. We should think of something which will replace this idiom.

Romain Beauxis added a comment - 31/Oct/11 12:51 PM This does not work: def f(x) =   print(x.foo)   print(x.bar) end At line 6, char 8:   this value has type     [foo:_] (infered at line 5, char 8-12)   but it should be a subtype of (the type of the value at line 6, char 8-12)     [bar:_]

David Baelde added a comment - 31/Oct/11 10:12 PM Yes, we need more clever inference, perhaps constraint based. Row variables may also be needed, for example fun(r)->[r with foo=1] should have type ([...] as 'a)->['a with foo:int].

Samuel Mimram added a comment - 01/Nov/11 2:00 AM Row polymorphism is there now :)

Samuel Mimram added a comment - 01/Nov/11 10:46 AM Note for later, replace_deep_field is still not working: Correct expression r = [a=[b=0]] r = [r with a = [r.a with t = 5.]] ignore (r)... r : [a:[b:int]] r : [a:[t:float, b:int]] Correct expression r = [a=[b=0]] def r.a.t = 5. end ignore (r)... r : [a:[b:int]] r : [a:[t:float]] The field b is removed in the second example...

Samuel Mimram added a comment - 04/Nov/11 4:32 PM Actually, it's even worse: # def f (r) = [r with a = 5] end;; f : ([, ?A])->[a:int, ?A] = <fun> # f ([b=3]);; - : [a:int, ?A] = [a = 5, b = 3]

Romain Beauxis added a comment - 05/Nov/11 1:31 AM So its fixed now, right? # def f(x) = [x with foo = 1] end;; f : ([, ?A])->[foo:int, ?A] = <fun> # f([bar=2]);; - : [foo:int, bar:int] = [foo = 1, bar = 2]

Samuel Mimram added a comment - 05/Nov/11 9:45 AM Yep, this is fixed now :)

Romain Beauxis added a comment - 07/Nov/11 1:32 AM Another issue:  let () =   let t = Lang.univ_t 1 in   add_builtin "record.has" ~cat:Liq     ~descr:"record.has(x,\"foo\") is true if field foo is defined in record x."    ["",Lang.record_t ~t [],None,None;     "",Lang.string_t,None,None] Lang.bool_t    (fun p ->      let l = Lang.to_record (Lang.assoc "" 1 p) in      let s = Lang.to_string (Lang.assoc "" 2 p) in      Lang.bool (List.mem_assoc s l)) # record.has;; - : ([, ?A],string)->bool = <fun> # record.has([foo = 1],"foo");; - : bool = true # record.has;; - : ([foo:int],string)->bool = <fun>

Romain Beauxis added a comment - 07/Nov/11 2:21 AM I have commited a branch off the branch (...) to add the following constructs: * ?(x.foo="foo") : fetch field foo or defaults to "foo" if not defined * x.foo? : true if x.foo is defined If working on the issue above, please consider first whether or not this can be merged first..

Romain Beauxis added a comment - 07/Nov/11 2:21 AM PS: toots@zulu src % ./liquidsoap ../scripts/list.liq ../scripts/http.liq -h http_response *** One entry in scripting values: Create a HTTP response callback when a data callback is provided. Category: Interaction Type: ([?protocol:string, ?code:int,   ?headers:[(string*string)], ?static:string,   ?stream:(()->string), 'a])->()->string Parameters: * (unlabeled) : [?protocol:string, ?code:int,  ?headers:[(string*string)], ?static:string,  ?stream:(()->string), 'a] (default: None)     Record containing all required informations. :-)

Romain Beauxis added a comment - 07/Nov/11 1:03 PM The above issue is not related to the "record.has" function: # def f(x) = [x with foo = 1] end;; f : ([, 'a])->[foo:int, 'a] = <fun> # f([gni=1.]);; - : [foo:int, gni:float] = [foo = 1, gni = 1.] # f;; - : ([gni:float])->[foo:int, gni:float] = <fun>

Samuel Mimram added a comment - 07/Nov/11 3:13 PM All this is expected since we don't have type scheme for record fields for now, and also I am maybe not generalizing row variables as I should: I am currently trying to implement all this, so please bear with it for now :)

Samuel Mimram added a comment - 09/Nov/11 5:06 PM Last thing to fix: # f = fun (r) -> [r with a = 4];; f : ([, 'a])->[a: int, 'a] = <fun> # r = record.empty;; r : [] = [] # r = f(r);; r : [a: int] = [a = 4] # r = f(r);; r : [a: int, a: int] = [a = 4] # r = f(r);; r : [a: int, a: int, a: int] = [a = 4]

Romain Beauxis added a comment - 09/Nov/11 5:40 PM I stand with David, why not using Map or Hashtbl for fields?

Samuel Mimram added a comment - 09/Nov/11 6:40 PM This is fixed, I now don't have anymore open bug concerning records.

Samuel Mimram added a comment - 09/Nov/11 6:41 PM Concerning the datastructure for fields, I was not so much concerned about performances but code simplicity. Due to the current uses of Liq I don't think that this really matters, but feel free to use maps if you prefer :)

Samuel Mimram added a comment - 09/Nov/11 6:53 PM Ah yes, one last thing: # list;; - : [length: ∀'a 'b 'a 'a 'a 'a 'a 'a 'i 'a 'i 'a 'a . ((['a])->int),      mem: ∀'a 'b 'a 'a 'a 'a 'a 'a 'i 'a 'i 'a 'a . (('b,['b])->bool),      rev: ∀'a 'b 'a 'a 'a 'a 'a 'a 'i 'a 'i 'a 'a . ((['a])->['a]),      ... I don't think that it comes from my code but from the generalized variables associated to builtins. Anyway, it's not incorrect (just very unnecessary...).

Samuel Mimram added a comment - 09/Nov/11 8:58 PM Actually, the last bug is also corrected now. The only left issue is that constraints are not well displayed for records (they should be displayed in the fields after the universal quantification, but I don't really have the motivation for this small thing right now.... Otherwise it works as expected.

Samuel Mimram added a comment - 17/Dec/11 4:18 PM Apparently my hack for record definition really has to be fixed: list = [ list with f = fun(~x=list.hd) -> ()] works without any problem but def list.f(~x=list.hd) = () end results in Fatal error: exception Not_found Raised at file "list.ml", line 144, characters 16-25 Called from file "lang/lang_values.ml", line 1055, characters 24-42 Called from file "lang/lang_values.ml", line 1091, characters 22-35 Called from file "tools/utils.ml", line 25, characters 51-56 Called from file "lang/lang_values.ml", line 1168, characters 44-67 Called from file "list.ml", line 57, characters 20-23 Called from file "lang/lang_values.ml", line 1167, characters 12-105 Called from file "lang/lang_values.ml", line 1123, characters 27-43 Called from file "lang/lang_values.ml", line 1250, characters 14-20 Called from file "lang/lang_values.ml", line 1315, characters 18-31 Called from file "lang/lang.ml", line 671, characters 14-38 Called from file "tools/tutils.ml", line 85, characters 14-18 Re-raised at file "tools/tutils.ml", line 85, characters 54-55 Called from file "lang/lang.ml", line 697, characters 8-12 Re-raised at file "lang/lang.ml", line 802, characters 49-50 Called from file "lang/lang.ml", line 815, characters 8-242 Called from file "lang/lang.ml", line 825, characters 4-76 Called from file "main.ml", line 108, characters 34-40 Called from file "main.ml", line 557, characters 2-29