Software Design Notes                            Flavor Internals







                           SECTION 13

                        Flavor Internals


13.1  INTRODUCTION

This  description  is  intended  for  programmers  familiar  with
Flavors;   this  is  not  introductory  material.   In  order  to
introduce the  principle  components  of  the  flavor  system,  a
simplified example follows:

 (DEFFLAVOR NEW-FLAVOR (IV1 IV2) (BASE) :GETTABLE-INSTANCE-VARIABLES)
 This macro expands into code that creates a FLAVOR STRUCTURE
 where most of the flavor internals structures are  stored. All
 the available information (Instance variables, dependencies,
 keywords) is put inthere.  This STRUCTURE is placed on the
 SYS:FLAVOR property of NEW-FLAVOR.

 (DEFMETHOD (NEW-FLAVOR :MESSAGE-1) (A) (+ A IV1))
 This macro expands into a function definition whose name is
 (:METHOD NEW-FLAVOR :MESSAGE-1). The method name and its
 function definition are going to be recorded in the METHOD-TABLE
 of NEW-FLAVOR.

 (MAKE-INSTANCE 'NEW-FLAVOR)
 This will first create the remaining flavor data structures
 needed for the instance to be functional (method-hash-table and
 mapping-table) (this is called composition).  Next, an instance
 of NEW-FLAVOR will be created and returned.

 (SEND INSTANCE :MESSAGE-1 7)
 This will cause the microcode to access the  flavor structure to
 get the function attached to :MESSAGE-1 [(:METHOD NEW-FLAVOR
 :MESSAGE-1) in that case] through the method-hash-table and call
 that method with the special variable SELF bound to instance.



13.2  FLAVOR NAMES

Flavor  names  are  symbols  which  get  interned  as usual.  The
SYS:FLAVOR  property  holds  the  complete  flavor   description,
contained  in a structure of type FLAVOR.  SYS:COMPILATION-FLAVOR
can be used to get the flavor structure from the symbol;  it  can
get  the temporary flavor structure used by the compiler during a
file compilation.



                             13-1

Flavor Internals                            Software Design Notes



13.3  FLAVOR INSTANCES

An Instance is an object with type DTP-INSTANCE that points to  a
block  of  memory  beginning  with  a  word of type DTP-INSTANCE-
HEADER.  The pointer field of the header  points  to  the  FLAVOR
STRUCTURE  (see  Flavor  Structure  section)  where the microcode
finds the METHOD-HASH-TABLE, the flavor name, etc.  The  function
(SYS:instance-flavor  <instance>) returns the flavor structure of
an instance.  The instance variables are stored right  after  the
header  word  and  can be accessed by offset using (%instance-ref
instance index).


        +--+-------------------+---------------------------+
        |CC|   DTP-Instance    |  Pointer to the instance  |
        +--+-------------------+---------------------------+
                                            |
     +--------------------------------------+
     |
     |  +--+-------------------+---------------------------+
     +->|CC|DTP-Instance-header|Pointer to Flavor structure|
        +--+-------------------+---------------------------+
        |          first instance variable                 |
        +--------------------------------------------------+
        |          second instance variable                |
        +--------------------------------------------------+
        |                   etc.                           |
        +--------------------------------------------------+


                  Figure 13-1  Flavor Instance




13.4  FLAVOR STRUCTURE

This structure is  used  at  DEFFLAVOR  time,  compile  time  and
runtime.   It is shared by all the instances of that flavor.  All
of  the  inter-Flavor  pointers  are   symbols,   not   structure
references,  this  allows  for  flavor  redefinition and renaming
without pointer repair.  Note that since the  structure  is  used
for  everything,  some slots are empty (NIL) if the flavor is not
fully composed.  In particular, if a structure has a  DEPENDS-ON-
ALL  slot of NIL, it is assumed that its components are yet to be
figured out.  If a structure has a METHOD-HASH-TABLE slot of NIL,
it is assumed  that  its  methods  are  not  composed  yet.   Its
description  is  included  in  Table  13-2, so the programmer can
figure out the accessor functions for each slot.






                             13-2

Software Design Notes                            Flavor Internals



13.5  FLAVOR METHOD-TABLE  (FLAVOR-METHOD-TABLE  slot  of  Flavor
structure)

This Flavor structure slot is used to store information about the
methods.   It  is  used  to  compose  the  methods  and to do the
function spec  handling  (for  FDEFINE,  FDEFINITION,  FDEFINEDP,
etc.).   For  the  previous simple example the value of this slot
would be:

 ((:MESSAGE-1
    NIL NIL
    ((:METHOD NEW-FLAVOR :MESSAGE-1)
     #<DTP-FEF-POINTER (:METHOD NEW-FLAVOR :MESSAGE-1) 20346350>
     NIL))
  (:IV2
    NIL NIL
    ((:METHOD NEW-FLAVOR :IV2)
     #<DTP-FEF-POINTER (:METHOD NEW-FLAVOR :IV2) 20346337>
     NIL))
  (:IV1
    NIL NIL
    ((:METHOD NEW-FLAVOR :IV1)
     #<DTP-FEF-POINTER (:METHOD NEW-FLAVOR :IV1) 20346326>
     NIL)))


Its format  is:   a  list  of  message  entries,  each  of  those
consisting   of    (MESSAGE  COMBINATION-TYPE  COMBINATION-OPTION
METH1 METH2 ....)  where:

      -  MESSAGE is the message symbol.

      -  COMBINATION-TYPE is  the  type  described  by  the
         DEFFLAVOR   declaration  :METHOD-COMBINATION  (for
         example, :CASE :PROGN ...).  If NIL, then  :DAEMON
         is used.

      -  COMBINATION-OPTION  is  either :BASE-FLAVOR-FIRST,
         :BASE-FLAVOR-LAST  or  NIL  (the   default)   (see
         DEFFLAVOR option).

      -  METHx   is   a   list  structure:   (FUNCTION-SPEC
         FUNCTION-OBJECT PLIST) where FUNCTION-SPEC is  the
         function-spec  of  the  method, FUNCTION-OBJECT is
         the method and PLIST is a property  list  used  to
         store    the   source   filename,   the   previous
         definition, etc.

The Method-Table is updated in two places:

      -  At DEFFLAVOR  time  when  there  is   a   :method-
         combination  keyword.   An  entry is made for that
         message, specifying the combination type  and  the


                             13-3

Flavor Internals                            Software Design Notes



         combination option.


      -  At  FDEFINE time   any Method Fdefined will get an
         entry in the table.  Note that  FDEFINE  will  not
         modify  the  combination  type and the combination
         option; by default they will be NIL.



13.6  INSTANCE VARIABLE ACCESS

There are three kinds of instances variables:

      -  SPECIALS.  These are special variables whose value
         cell is forwarded to  the  correct  place  in  the
         instance.  The binding is done by the microcode at
         SEND  time.  These variables are produced by using
         the DEFFLAVOR keyword :SPECIAL-INSTANCE-VARIABLES.


      -  UNMAPPED:  These variables are assumed to  have  a
         fixed  place  in the instance even when mixed with
         other  components.    Therefore,   they   can   be
         referenced  by direct offset in any method.  These
         variables are  produced  by  using  the  DEFFLAVOR
         keyword :ORDERED-INSTANCE-VARIABLES.


      -  MAPPED:   These  variables  are not going to be at
         the same  place  from  one  flavor  to  the  other
         because  they are mixed with different components.
         However, their methods (except  combined  methods)
         are  expected  to work on any flavor that includes
         it.  The microcode  gets  the  offset  within  the
         instance  through  a  mapping  table,  which is an
         array.  Each instance variable is addressed  using
         its  corresponding  mapping  table  index, and the
         array element at that index is the offset  of  the
         instance  variable  within  the  instance.   These
         variables are produced by default.



13.7  FLAVOR MAPPING TABLES

Mapping table structure:  It is an  array  with  a  fill-pointer.
The  length  of  the  array  is  the  number  of  mapped instance
variables and each element of the array  is  the  offset  of  the
instance  variable  within  the instance.  When a combined method
calls a method that comes from one of its component flavors, then
it must pass another mapping table as  well.   The  reference  to
that  mapping  table  is  contained  in the leader of the primary


                             13-4

Software Design Notes                            Flavor Internals



mapping table starting at array-leader index 3.

                    Table 13-1  Mapping Table


            +----------------------+    \
            |mapping-table-locative|     |
            | for component n      | (+ n 3)
            +----------------------+     |
            |         .            |     |
            |         .            |     |
            +----------------------+     |
            |mapping-table-locative|     |
            | for component 0      |  3   > Array leader
            +----------------------+     |
            |   Instance flavor    |  2  |
            |  structure pointer   |     |
            +----------------------+     |
            |    Method flavor     |  1  |
            |  structure pointer   |     |
            +----------------------+     |
            | fill pointer         |  0  |
            +----------------------+    /
            +----------------------+
            |  offset #0           |
            +----------------------+
            |         .            |
            |         .            |
            +----------------------+
            |  offset #n           |
            +----------------------+



These mapping tables are found:

      -  In  the  METHOD-HASH-TABLE  (see  section   FLAVOR
         METHOD HASH TABLE)


      -  In  the  COMPONENT-MAPPING-TABLE-ALIST slot of the
         flavor structure.  This an alist whose key is  the
         method-flavor  and  cdr is a locative to the right
         mapping table.  It is used  by  GET-MAPPING-TABLE-
         LOCATION when building the method-hash-table.


      -  In   the   FLAVOR-COMPONENT-MAPPING-VECTOR   which
         gathers them all, the contents of this accessor is
         an ART-Q-LIST vector containing locatives  to  the
         mapping tables.




                             13-5

Flavor Internals                            Software Design Notes



The  mapping  tables  are created when GET-MAPPING-TABLE-LOCATION
finds out that they do  not  exist.   So  it  will  occur  during
execution of COMPOSE-METHOD-COMBINATION.



13.8  FLAVOR  METHOD HASH TABLE (FLAVOR-METHOD-HASH-TABLE slot of
Flavor structure

The method-hash-table is used mainly by the microcode  to  figure
out  what  function  to  call  for  a given message.  The FLAVOR-
METHOD-HASH-TABLE slot points to a hash array whose  key  is  the
message  symbol and values are:  a locative to a cell that points
to the method object and a pointer to the mapping  table.   Right
mow the hash function is just a LOGAND mask on the symbol pointer
(as a consequence the hash-table size has to be an exact power of
two).   If the hash fails, the function SYS:INSTANCE-HASH-FAILURE
is called from the microcode.  That function will figure  out  if
the hash table needs to be rehashed and if there were any forward
references from the flavor structure slot to the real hash array.
If  this  is  the case, the forwarding pointer is snapped out and
the function will retry.  That prevents  from  having  to  follow
slow forwarding pointers and not knowing about them.

Note  that  the FLAVOR-METHOD-HASH-TABLE slot is filled in at the
last moment, therefore, it is essentially used at  runtime.   Any
type  of  method lookup will use the method table instead because
it is always up to date.



13.9  FLAVOR COMPOSITION (COMPOSE-FLAVOR-COMBINATION)

This function is invoked either at COMPILE-FLAVOR-METHODS time or
at instantiation time.  It must be completed  before  the  Method
Composition  can  take  place.   This  operation looks at all the
components of a flavor and if they are defined, finds  out  about
all  the  instance  variables  that  make up that instance.  This
operation must be completed  before  attempting  to  compose  the
methods.   Upon  its  completion, the slot DEPENDS-ON-ALL will be
non-null.  The first step is COMPOSE-FLAVOR-INCLUSION which  goes
through  the flavor components and returns an ordered list of all
the components.  Then the instance variables are gathered, taking
into account the ordered instance variables.



13.10  METHOD COMPOSITION (COMPOSE-METHOD-COMBINATION)

This function is called right after  the  Flavor  Composition  is
complete.   This  operation will figure out, for each message the
right method to invoke.   If  it  requires  creating  a  combined
method,  it  will  do  so.  Upon its completion, the slot FLAVOR-


                             13-6

Software Design Notes                            Flavor Internals



METHOD-HASH-TABLE will be non-null.  First the  "magic  list"  is
created: it is similar to the FLAVOR-METHOD-TABLE but gathers all
the  non  combined  methods  from all the component flavors.  The
message orderign can be altered by the DEFFLAVOR keyword :METHOD-
ORDER.  Within each message, methods are inserted so the  default
order  is  base-flavor-last.   Once that magic list is built, any
:default method that does not need to be there is  removed,  then
the  method  combination  is  done  for each message.  The method
combination  handler  (located  on   the   SYS:METHOD-COMBINATION
property)  is  invoked  on  the  message  sublist,  returning the
combined method for that  message  (see  the  Method  Combination
section).   Next,  the  FLAVOR-METHOD-HASH-TABLE  is  updated  or
created putting  the  method  and  the  mapping  tables  for  the
messages  in  the  hash table.  The mapping tables are created at
that time if they did not exist before.



13.11  METHOD COMBINATION

The handlers for method combination are called by COMPOSE-METHOD-
COMBINATION with the flavor and the  magic  list  siblist  for  a
given  message.  Its role is to return a method for the specified
flavor and message.  It  will  create  the  combined  methods  as
necessary.   The  handlers  select  the daemon they need (:before
:after ...),  then  they  check  the  combined  method  with  the
function  HAVE-COMBINED-METHOD.   If it returns a method, then it
means that there was an up-to-date combined method  and  that  is
the  handler.   If  it  returns NUL, then MAKE-COMBINED-METHOD is
called and will create one.



13.12  HAVE-COMBINED-METHODS

This function will try to get a Combined method that matches  the
magic  list  for that operation by getting the combined method of
each of the component flavors starting in BASE-FLAVOR-LAST order.
The magic list  used  to  create  a  combined  method  is  either
recorded in the COMBINED-METHOD-DERIVATION debug info field or in
the function's plist.



13.13  INSTANCE VARIABLE ADDRESSING

The   compiler  produces  symbolic  references  to  the  instance
variables.  At load time, the function  FLAVOR-VAR-SELF-REF-INDEX
is  called and will return a fixnum that represents an offset and
a flag, %%SELF-REF-RELOCATE-FLAG, that tells the microcode if  it
should  go  through  the  mapping table or not.  If the offset is
small enough (< 32)  then  the  SLEF  addressing  mode  is  used,
otherwise  a FEF-constant is used which will have a DTP-SELF-REF-


                             13-7

Flavor Internals                            Software Design Notes



POINTER data type and its pointer field will be the offset.   The
value  of  the  offset  will  be  the  position  of that instance
variable in  the  FLAVOR-UNMAPPED-INSTANCE-VARIABLES  or  FLAVOR-
MAPPED-INSTANCE-VARIABLES slot of the structure.  If the variable
is  not  present  in  either  of  those, it is assumed mapped and
pushed on FLAVOR-MAPPED-INSTANCE-VARIABLES.  The  mapping  tables
are  updated if they exist, otherwise they will be created during
execution of COMPOSE-METHOD-COMBINATION.



13.14  MAPPING TABLE REFERENCE

When a combined method calls a method for one  of  its  component
flavors, it passes a new mapping table for that component flavor.
The  DTP-SELF-REF-POINTER  addressing  mode  is  used, the offset
represents the array-leader  index  (see  how  component  mapping
tables  are referenced from the main mapping table in the mapping
table section).  A flag, %%self-ref-map-leader-flag,  is  set  to
distinguish  between  a  mapping  table reference and an instance
variable reference.



13.15  METHOD INTEGRATION

In order to reduce  combined  method  overhead,  when  the  SPEED
compiler  switch  is greater than the SAFETY compiler switch, the
variable SYS:*INTEGRATE-COMBINED-METHODS* is set  to  T  and  the
compiler will expand the daemons inline during execution of MAKE-
COMBINED-METHOD  (this  function  calls OPTIMIZE-METHOD-BODY-AND-
ARGS which does the expansion based  on  SYS:*INTEGRATE-COMBINED-
METHODS*.



13.16  COMPILE FLAVOR METHODS

When Compiling to a file, this macro will cause the creation of a
new Flavor Structure in the compiler's environment and a COMPOSE-
FLAVOR-COMPOSITION and COMPOSE-METHOD-COMPOSITION is performed on
it.   During  the  execution  of  COMPOSE-METHOD-COMPOSITION, any
combined method being created will be dumped on the  object  file
as  a  side effect.  At load time, it will insure that the flavor
is fully composed, checking the FLAVOR-DEPENDS-ON-ALL and FLAVOR-
METHOD-HASH-TABLE slots.



13.17  DEFFLAVOR

The flavor structure is created at this point,  with  many  slots
set  to  NUL.  FLAVOR-DEPENDS-ON-ALL and FLAVOR-METHOD-HASH-TABLE


                             13-8

Software Design Notes                            Flavor Internals



are set to NIL so that  MAKE-INSTANCE  or  COMPILE-FLAVOR-METHODS
will know they have to compose that flavor.



13.18  DEFMETHOD

This  macro  will  produce  a  function object that gets a :self-
                                                           ______
flavor declaration to declare which variables  are  going  to  be
______
instance variables.



13.19  METHOD LOADING

When a method is loaded or fdefined, METHOD-FUNCTION-SPEC-HANDLER
is  invoked and will insert the method description in the METHOD-
TABLE slot (see METHOD  TABLE  section).   Any  :FASLOAD-COMBINED
type  of  method  will be turned into a :COMBINED method and will
supersede the old definition only if the old  definition  is  not
FEF-EQUAL  to  the  new one; this is due to the fact that without
method integration, the combined methods do not contain user code
by themselves but only calls to  the  appropriate  non-  combined
methods.   Thus  if the flavor composition did not change the old
combined method, it is still valid even if  the  individual  non-
combined methods were changed.



13.20  MAKE-INSTANCE (INSTANTIATE-FLAVOR)

INSTANTIATE-FLAVOR will get the real flavor structure (chasing it
through  abstract  flavors),  get  the  area to cons in using the
function stored in the  INSTANCE-AREA-FUNCTION  property  of  the
flavor  plist,  check that the flavor is composed (DEPENDS-ON-ALL
non-null), check that the methods are composed (METHOD-HASH-TABLE
non-null) then calls the %ALLOCATE-AND-INITIALIZE-INSTANCE miscop
that creates the instance structure with the  instance  variables
set  to DTP-NULL, then the instance variables will be initialized
to values coming  from  the  init-plist  and  the  default-values
defined at DEFFLAVOR time.  Finally, the :INIT message is sent.



12.21  INSTANCE CALLING (SEND)

The  microcode  is responsible for the instance calling.  It gets
the method from the  METHOD-HASH-TABLE  (See  Method  Hash  Table
section).   If  the  <mapping-table-flag>  is set for that method
then it will look for a mapping table as the second value in  the
hash  entry,  if  it  not  there,  it  will lookup the COMPONENT-
MAPPING-TABLE-ALIST.  Given  that,  it  will  bind  SELF  to  the
instance,  SELF-MAPPING-TABLE  to  the mapping table and call the


                             13-9

Flavor Internals                            Software Design Notes



method.






















































                            13-10

Software Design Notes                            Flavor Internals




             Table 13-2  Flavor Strucure Definition

(defstruct (FLAVOR :named :array (:constructor make-flavor)
                   (:alterant nil)
                   (:make-array (:area permanent-storage-area))
                   (:conc-name nil) (:callable-constructors nil)
                   (:predicate nil) (:copier nil))

  flavor-instance-size       ;1+ the number of instance
                             ;  variables
  flavor-bindings            ;List of locatives to instance
                             ;  variable internal value cells.
                             ;  MUST BE CDR-CODED!!
                             ;Fixnums can also appear.  They
                             ;  say to skip whatever number of
                             ;  instance variable slots.
  FLAVOR-METHOD-HASH-TABLE   ;The hash table for methods of
                             ;  this flavor.  NIL -
                             ;  method-combination not composed
                             ;  yet.  T means abstract flavor
                             ;  with COMPILE-FLAVOR-METHODS done.
  FLAVOR-NAME                ;Symbol which is the name of the
                             ;  flavor.  This is returned by
                             ;  TYPEP.

  FLAVOR-COMPONENT-MAPPING-TABLE-ALIST  ;Alist of component
                                        ;  flavor names vs.
                                        ;  locatives into vector
                                        ;  containing mapping
                                        ;  tables.

  ;; End of magic locations known in microcode and QCOM.

  flavor-local-instance-variables  ;Names and initializations,
                                   ;  does not include inherited
                                   ;  ones.
  flavor-all-instance-variables    ;Just names, only valid when
                                   ;  flavor-combination composed.
                                   ;Corresponds directly to
                                   ;  FLAVOR-BINDINGS and the
                                   ;  instances.
  FLAVOR-METHOD-TABLE              ;Defined above.

  ;; End of locations depended on in many other files.

  FLAVOR-DEPENDS-ON              ;List of names of flavors
                                 ;  incorporated into
                                 ;  this flavor.
  flavor-depended-on-by          ;List of names of flavors
                                 ;  which incorporate this one.
                                 ;The above are only immediate
                                    dependencies.


                            13-11

Flavor Internals                            Software Design Notes



  flavor-includes                ;List of names of flavors to
                                 ;  include at the end rather
                                 ;  than as immediate depends-on's.
  flavor-package                 ;Package in which the DEFFLAVOR
                                 ;  was done.
  FLAVOR-DEPENDS-ON-ALL          ;Names of all flavors depended on,
                                 ;  to all levels, including this
                                 ;  flavor itself.  NIL means
                                 ;  flavor-combination not
                                 ;  composed yet.  This is used by
                                 ;  TYPEP of 2 arguments.
  (flavor-which-operations ())   ;List of operations handled, created
                                 ;  when needed.  This is NIL if it
                                 ;  has not been computed yet.

  ;;This is the list of instance variables accessible from
  ;;this flavor which are mapped by mapping tables with this
  ;;flavor as the method-flavor.

  (flavor-mapped-instance-variables ())

  ;; Redundant copy of :DEFAULT-HANDLER property,
  ;;for speed in calling it.

  (flavor-default-handler ())
  (flavor-inittable-instance-variables ())   ;Alist from init
                                             ;  keyword to name
                                             ;  of variable
  (flavor-init-keywords ())    ;option
  (flavor-plist ())            ;Esoteric things stored here as
                              ;  properties (see Table 13-3)
          )


               Table 13-3  Flavor-Plist Properties

:ORDERED-INSTANCE-VARIABLES
:DEFAULT-HANDLER
:OUTSIDE-ACCESSIBLE-INSTANCE-VARIABLES
:ACCESSOR-PREFIX
:REQUIRED-INSTANCE-VARIABLES
:REQUIRED-METHODS
:REQUIRED-FLAVORS
:SELECT-METHOD-ORDER
:DEFAULT-INIT-PLIST
:DOCUMENTATION:NO-VANILLA-FLAVOR
:GETTABLE-INSTANCE-VARIABLES
:SETTABLE-INSTANCE-VARIABLES
:SPECIAL-INSTANCE-VARIABLES
:ABSTRACT-FLAVOR
:ALIAS-FLAVOR
:INSTANTIATION-FLAVOR-FUNCTION
:RUN-TIME-ALTERNATIVES or


                            13-12

Software Design Notes                            Flavor Internals



:MIXTURE
SYS:RUN-TIME-ALTERNATIVE-ALIST  is the alist of lists of flavors
        vs. names we constructed for those combinations.
SYS:ADDITIONAL-INSTANCE-VARIABLES
SYS:COMPILE-FLAVOR-METHODS
SYS:UNMAPPED-INSTANCE-VARIABLES
SYS:MAPPED-COMPONENT-FLAVORS
SYS:ALL-INSTANCE-VARIABLES-SPECIAL
SYS:INSTANCE-VARIABLE-INITIALIZATIONS
SYS:ALL-INITTABLE-INSTANCE-VARIABLES
SYS:REMAINING-DEFAULT-PLIST
SYS:REMAINING-INIT-KEYWORDS
:INSTANCE-AREA-FUNCTION  the one specified for this fl.
SYS:INSTANCE-AREA-FUNCTION the one to be used (maybe inherited)
:REQUIRED-INIT-KEYWORDS the ones specified for this fl.
SYS:REQUIRED-INIT-KEYWORDS  all required ones incl. inherited.


                              NOTE

          The slots stored  as  properties  of  FLAVOR-
          PLIST  have  accessor  functions  of the form
          FLAVOR-<property-name>, like FLAVOR-UNMAPPED-
          INSTANCE-VARIABLES.

          The convention on these  is  supposed  to  be
          that ones in the keyword packages are allowed
          to  be  used by users.  Some of these are not
          used by the  flavor  system,  they  are  just
          remembered on the plist in case anyone cares.
          The  flavor  system  does all its handling of
          them during the expansion  of  the  DEFFLAVOR
          macro.






















                            13-13