Tensor Is the Might

(zserge.com)

25 points | by eatonphil 1 hour ago

2 comments

  • srean 51 minutes ago
    If one wants to add the capability to reason about shape and shape compatibility, Barry Jay's FiSh would be an interesting detour.

    https://web.archive.org/web/20111015133833/http://www-staff....

    This was used in his shape aware language FiSh, for dealing with multidimensional arrays. Shape compatibilities were statically type checked, if I recall correctly. Shapes were also used to optimize the loops.

    [Programming in FISh] https://link.springer.com/article/10.1007/s100090050037

    [Towards Dynamic Shaping] https://www.researchgate.net/publication/265975794_Towards_D...

    • zserge 48 minutes ago
      I wish we used something else but Python as the default ML language, so that some advanced type system would work for us to ensure tensor shapes and compatibility.
      • churchsub1 11 minutes ago
        Here's a Python library for static type checking of tensor shapes: https://pypie.dev/
      • srean 36 minutes ago
        Honestly, I quite like Python most of the time. The times when I don't, argghh type errors.

        The worst I have had in a while is when I was dealing with a sparse array data type and getting some processing done in a C function to speed things up.

        Infuriatingly enough, the index type used to change after initialising the array. It would check if Int32 could accomodate the size of the array. If it could it would downgrade to Int32 at runtime. My C counterpart had no clue.

        That was so not fun to debug because the errors were intermittent (depending on the fill factor of the sparse array)

      • weiserwx 26 minutes ago
        [dead]
  • srean 1 hour ago
    > A tensor is nothing but a flat array of numbers, plus some metadata telling you how to interpret those numbers as a multi-dimensional object.

    Erm... many would disagree. I think what he means is just a multidimensional array.

    • zserge 1 hour ago
      Not in a general mathematical sense, of course. Rather carthesian tensors. Seems like for most libraries it's `numpy.ndarray`: https://docs.tinygrad.org/tensor/ or https://docs.pytorch.org/docs/main/tensors.html
      • srean 54 minutes ago
        To add to the confusion, Mathematicians and Physicists often disagree on what that word means. A Physicist's tensor is often a mathematicians tensor field.

        An appropriate tensor can certainly be represented by a multidimensional array but not all multidimensional arrays are tensors, it depends on how that multidimensional array participates with others.

        For example, (lat, long) is not a 2d vector although it can be represented as a 2 dimensional array.