15

I'm looking for a way to have all keys / values pair of a nested object.

(For the autocomplete of MongoDB dot notation key / value type)

interface IPerson {
    name: string;
    age: number;
    contact: {
        address: string;
        visitDate: Date;
    }
}

Here is what I want to achieve, to make it becomes:

type TPerson = {
    name: string;
    age: number;
    contact: { address: string; visitDate: Date; }
    "contact.address": string;
    "contact.visitDate": Date;
}

What I have tried:

In this answer, I can get the key with Leaves<IPerson>. So it becomes 'name' | 'age' | 'contact.address' | 'contact.visitDate'.

And in another answer from @jcalz, I can get the deep, related value type, with DeepIndex<IPerson, ...>.

Is it possible to group them together, to become type like TPerson?

Modified 9/14: The use cases, need and no need:

When I start this question, I was thinking it could be as easy as something like [K in keyof T]: T[K];, with some clever transformation. But I was wrong. Here is what I need:

1. Index Signature

So the interface

interface IPerson {
    contact: {
        address: string;
        visitDate: Date;
    }[]
}

becomes

type TPerson = {
    [x: `contact.${number}.address`]: string;
    [x: `contact.${number}.visitDate`]: Date;
    contact: {
        address: string;
        visitDate: Date;
    }[];
}

No need to check for valid number, the nature of Array / Index Signature should allow any number of elements.

2. Tuple

The interface

interface IPerson {
    contact: [string, Date]
}

becomes

type TPerson = {
    [x: `contact.0`]: string;
    [x: `contact.1`]: Date;
    contact: [string, Date];
}

Tuple should be the one which cares about valid index numbers.

3. Readonly

readonly attributes should be removed from the final structure.

interface IPerson {
    readonly _id: string;
    age: number;
    readonly _created_date: Date;
}

becomes

type TPerson = {
    age: number;
}

The use case is for MongoDB, the _id, _created_date cannot be modified after the data has been created. _id: never is not working in this case, since it will block the creation of TPerson.

4. Optional

interface IPerson {
    contact: {
        address: string;
        visitDate?: Date;
    }[];        
}

becomes

type TPerson = {
    [x: `contact.${number}.address`]: string;
    [x: `contact.${number}.visitDate`]?: Date;
    contact: {
        address: string;
        visitDate?: Date;
    }[];
}

It's sufficient just to bring the optional flags to transformed structure.

5. Intersection

interface IPerson {
    contact: { address: string; } & { visitDate: Date; }
}

becomes

type TPerson = {
    [x: `contact.address`]: string;
    [x: `contact.visitDate`]?: Date;
    contact: { address: string; } & { visitDate: Date; }
}

6. Possible to Specify Types as Exception

The interface

interface IPerson {
    birth: Date;
}

becomes

type TPerson = {
    birth: Date;
}

not

type TPerson = {
    age: Date;
    "age.toDateString": () => string;
    "age.toTimeString": () => string;
    "age.toLocaleDateString": {
    ...
}

We can give a list of Types to be the end node.

Here is what I don't need:

  1. Union. It could be too complex with it.
  2. Class related keyword. No need to handle keywords ex: private / abstract .
  3. All the rest I didn't write it here.
13
  • Oof these "flatten subproperties" questions never seem to consider edge cases enough and they drive me spiraling into madness. Based on what principle would you decide not to go drilling way down into Date here? See this code. What would you like to change here? How do I know that Date should just be Date but other object types should be flattened?
    – jcalz
    Sep 10, 2021 at 3:47
  • I suppose I could hardcode Date as an exception, like this, but that's pretty ad-hoc.
    – jcalz
    Sep 10, 2021 at 3:51
  • 1
    Do you care about arrays? Like this? There are just so many caveats.
    – jcalz
    Sep 10, 2021 at 3:56
  • 1
    I mean, there are loads of potential edge cases: index signatures, readonly properties, optional properties, pattern template literal keys... I'm happy to write up what I have, but the failure modes of this thing could fill a book. Ideally you would tell me the sorts of things you need to support; sounds like arrays are important (which have index signatures)... anything else?
    – jcalz
    Sep 10, 2021 at 13:30
  • 1
    @jcalz I modified my question, with a list of use cases. Your sample code helped me so much than other answers. Before I think over it carefully, I almost think it's already a perfect answer. I'm busy working in my daily life currently, you get me back to the rail to ask a good question =)
    – Val
    Sep 14, 2021 at 4:10

2 Answers 2

9
+200

Below is the full implementation I have of Flatten<T, O> which transforms a type possibly-nested T into a "flattened" version whose keys are the dotted paths through the original T. The O type is an optional type where you can specify a (union of) object type(s) to leave as-is without flattening them. In your example, this is just Date, but you could have other types.

Warning: it's hideously ugly and probably fragile. There are edge cases all over the place. The pieces that make it up involve weird type manipulations that either don't always do what one might expect, or are impenetrable to all but the most seasoned TypeScript veterans, or both.

In light of that, there is no such thing as a "canonical" answer to this question, other than possibly "please don't do this". But I'm happy to present my version.

Here it is:


type Flatten<T, O = never> = Writable<Cleanup<T>, O> extends infer U ?
    U extends O ? U : U extends object ?
    ValueOf<{ [K in keyof U]-?: (x: PrefixKeys<Flatten<U[K], O>, K, O>) => void }>
    | ((x: U) => void) extends (x: infer I) => void ?
    { [K in keyof I]: I[K] } : never : U : never;

The basic approach here is to take your T type, and return it as-is if it's not an object or if it extends O. Otherwise, we remove any readonly properties, and transform any arrays or tuples into a version without all the array methods (like push() and map()) and get U. We then flatten each property in that. We have a key K and a flattened property Flatten<U[K]>; we want to prepend the key K to the dotted paths in Flatten<U[K]>, and when we're done with all that we want to intersect these flattened objects (with the unflattened object too) all together to be one big object.

Note that convincing the compiler to produce an intersection involves conditional type inference in contravariant positions (see Transform union type to intersection type), which is where those (x: XXX) => void) and extends (x: infer I) => void pieces come in. It makes the compiler take all the different XXX values and intersect them to get I.

And while an intersection like {foo: string} & {bar: number} & {baz: boolean} is what we want conceptually, it's uglier than the equivalent {foo: string; bar: number; baz: boolean} so I do some more conditional type mapping with { [K in keyof I]: I[K] } instead of just I (see How can I see the full expanded contract of a Typescript type?).

This code generally distributes over unions, so optional properties may end up spawning unions (like {a?: {b: string}} could produce {"a.b": string; a?: {b: string}} | {"a": undefined, a?: {b: string}}, and while this might not be the representation you were going for, it should work (since, for example, "a.b" might not exist as a key if a is optional).


The Flatten definition depends on helper type functions that I will present here with various levels of description:

type Writable<T, O> = T extends O ? T : {
    [P in keyof T as IfEquals<{ [Q in P]: T[P] }, { -readonly [Q in P]: T[P] }, P>]: T[P]
}

type IfEquals<X, Y, A = X, B = never> =
    (<T>() => T extends X ? 1 : 2) extends
    (<T>() => T extends Y ? 1 : 2) ? A : B;

The Writable<T, O> returns a version of T with the readonly properties removed (unless T extends O in which case we leave it alone). It comes from TypeScript conditional types - filter out readonly properties / pick only required properties.

Next:

type Cleanup<T> =
    0 extends (1 & T) ? unknown :
    T extends readonly any[] ?
    (Exclude<keyof T, keyof any[]> extends never ?
        { [k: `${number}`]: T[number] } : Omit<T, keyof any[]>) : T;

The Cleanup<T> type turns the any type into the unknown type (since any really fouls up type manipulation), turns tuples into objects with just individual numericlike keys ("0" and "1", etc), and turns other arrays into just a single index signature.

Next:

type PrefixKeys<V, K extends PropertyKey, O> =
    V extends O ? { [P in K]: V } : V extends object ?
    { [P in keyof V as
        `${Extract<K, string | number>}.${Extract<P, string | number>}`]: V[P] } :
    { [P in K]: V };

PrefixKeys<V, K, O> prepends the key K to the path in V's property keys... unless V extends O or V is not an object. It uses template literal types to do so.

Finally:

type ValueOf<T> = T[keyof T]

turns a type T into a union of its properties. See Is there a `valueof` similar to `keyof` in TypeScript?.

Whew! 😅


So, there you go. You can verify how closely this conforms to your stated use cases. But it's very complicated and fragile and I wouldn't really recommend using it in any production code environment without a lot of testing.

Playground link to code

4
  • Excellent answer! It's already elegant enough to me. Thank you!
    – Val
    Sep 19, 2021 at 17:48
  • Thanks for the elegant solution! I wonder if there is a way to improve it to infer the type of the value of the nested property? I've tried to do some simple manipulations with your code, but it didn't work out as I would like it to. lookupWithReturn(foo, "a.0", 20) // OK lookupWithReturn(foo, "a.0", "20") // ERROR! string !== number lookupWithReturn(foo, "a.0", false) // ERROR! boolean !== number Dec 30, 2022 at 8:09
  • It would be awesome if you could answer my question, so I will be able to give you bounty (I plan to add it in a couple of days, when I can): stackoverflow.com/questions/74956397/… Dec 30, 2022 at 8:11
  • @MarkDolbyrev Hmm, well you deleted that question, and I had already commented and suggested this playground link as a possible approach. If that works for you I'd be happy to write up an answer explaining it... as long as the question is not deleted, of course. Not sure what else I can do here.
    – jcalz
    Dec 30, 2022 at 16:03
8

In order to achieve this goal we need to create permutation of all allowed paths. For example:

type Structure = {
    user: {
        name: string,
        surname: string
    }
}

type BlackMagic<T>= T

// user.name | user.surname
type Result=BlackMagic<Structure>

Problem becomes more interesting with arrays and empty tuples.

Tuple, the array with explicit length, should be managed in this way:

type Structure = {
    user: {
        arr: [1, 2],
    }
}

type BlackMagic<T> = T

// "user.arr" | "user.arr.0" | "user.arr.1"
type Result = BlackMagic<Structure>

Logic is straitforward. But how we can handle number[]? There is no guarantee that index 1 exists.

I have decided to use user.arr.${number}.

type Structure = {
    user: {
        arr: number[],
    }
}

type BlackMagic<T> = T

// "user.arr" | `user.arr.${number}`
type Result = BlackMagic<Structure>

We still have 1 problem. Empty tuple. Array with zero elements - []. Do we need to allow indexing at all? I don't know. I decided to use -1.

type Structure = {
    user: {
        arr: [],
    }
}

type BlackMagic<T> = T

//  "user.arr" | "user.arr.-1"
type Result = BlackMagic<Structure>

I think the most important thing here is some convention. We can also use stringified `"never". I think it is up to OP how to handle it.

Since we know how we need to handle different cases we can start our implementation. Before we continue, we need to define several helpers.

type Values<T> = T[keyof T]
{
    // 1 | "John"
    type _ = Values<{ age: 1, name: 'John' }>
}

type IsNever<T> = [T] extends [never] ? true : false;
{
    type _ = IsNever<never> // true 
    type __ = IsNever<true> // false
}

type IsTuple<T> =
    (T extends Array<any> ?
        (T['length'] extends number
            ? (number extends T['length']
                ? false
                : true)
            : true)
        : false)
{
    type _ = IsTuple<[1, 2]> // true
    type __ = IsTuple<number[]> // false
    type ___ = IsTuple<{ length: 2 }> // false
}

type IsEmptyTuple<T extends Array<any>> = T['length'] extends 0 ? true : false
{
    type _ = IsEmptyTuple<[]> // true
    type __ = IsEmptyTuple<[1]> // false
    type ___ = IsEmptyTuple<number[]> // false

}

I think naming and tests are self explanatory. At least I want to believe :D

Now, when we have all set of our utils, we can define our main util:

/**
 * If Cache is empty return Prop without dot,
 * to avoid ".user"
 */
type HandleDot<
    Cache extends string,
    Prop extends string | number
    > =
    Cache extends ''
    ? `${Prop}`
    : `${Cache}.${Prop}`

/**
 * Simple iteration through object properties
 */
type HandleObject<Obj, Cache extends string> = {
    [Prop in keyof Obj]:
    // concat previous Cacha and Prop
    | HandleDot<Cache, Prop & string>
    // with next Cache and Prop
    | Path<Obj[Prop], HandleDot<Cache, Prop & string>>
}[keyof Obj]

type Path<Obj, Cache extends string = ''> =
    // if Obj is primitive
    (Obj extends PropertyKey
        // return Cache
        ? Cache
        // if Obj is Array (can be array, tuple, empty tuple)
        : (Obj extends Array<unknown>
            // and is tuple
            ? (IsTuple<Obj> extends true
                // and tuple is empty
                ? (IsEmptyTuple<Obj> extends true
                    // call recursively Path with `-1` as an allowed index
                    ? Path<PropertyKey, HandleDot<Cache, -1>>
                    // if tuple is not empty we can handle it as regular object
                    : HandleObject<Obj, Cache>)
                // if Obj is regular  array call Path with union of all elements
                : Path<Obj[number], HandleDot<Cache, number>>)
            // if Obj is neither Array nor Tuple nor Primitive - treat is as object    
            : HandleObject<Obj, Cache>)
    )

// "user" | "user.arr" | `user.arr.${number}`
type Test = Extract<Path<Structure>, string>

There is small issue. We should not return highest level props, like user. We need paths with at least one dot.

There are two ways:

  • extract all props without dots
  • provide extra generic parameter for indexing the level.

Two options are easy to implement.

Obtain all props with dot (.):

type WithDot<T extends string> = T extends `${string}.${string}` ? T : never

While above util is readable and maintainable, second one is a bit harder. We need to provide extra generic parameter in both Path and HandleObject. See this example taken from other question / article:

type KeysUnion<T, Cache extends string = '', Level extends any[] = []> =
  T extends PropertyKey ? Cache : {
    [P in keyof T]:
    P extends string
    ? Cache extends ''
    ? KeysUnion<T[P], `${P}`, [...Level, 1]>
    : Level['length'] extends 1 // if it is a higher level - proceed
    ? KeysUnion<T[P], `${Cache}.${P}`, [...Level, 1]>
    : Level['length'] extends 2 // stop on second level
    ? Cache | KeysUnion<T[P], `${Cache}`, [...Level, 1]>
    : never
    : never
  }[keyof T]

Honestly, I don't think it will be easy for any one to read this.

We need to implement one more thing. We need to obtain a value by computed path.


type Acc = Record<string, any>

type ReducerCallback<Accumulator extends Acc, El extends string> =
    El extends keyof Accumulator ? Accumulator[El] : Accumulator

type Reducer<
    Keys extends string,
    Accumulator extends Acc = {}
    > =
    // Key destructure
    Keys extends `${infer Prop}.${infer Rest}`
    // call Reducer with callback, just like in JS
    ? Reducer<Rest, ReducerCallback<Accumulator, Prop>>
    // this is the last part of path because no dot
    : Keys extends `${infer Last}`
    // call reducer with last part
    ? ReducerCallback<Accumulator, Last>
    : never

{
    type _ = Reducer<'user.arr', Structure> // []
    type __ = Reducer<'user', Structure> // { arr: [] }
}

You can find more information about using Reducein my blog.

Whole code:

type Structure = {
    user: {
        tuple: [42],
        emptyTuple: [],
        array: { age: number }[]
    }
}


type Values<T> = T[keyof T]
{
    // 1 | "John"
    type _ = Values<{ age: 1, name: 'John' }>
}

type IsNever<T> = [T] extends [never] ? true : false;
{
    type _ = IsNever<never> // true 
    type __ = IsNever<true> // false
}

type IsTuple<T> =
    (T extends Array<any> ?
        (T['length'] extends number
            ? (number extends T['length']
                ? false
                : true)
            : true)
        : false)
{
    type _ = IsTuple<[1, 2]> // true
    type __ = IsTuple<number[]> // false
    type ___ = IsTuple<{ length: 2 }> // false
}

type IsEmptyTuple<T extends Array<any>> = T['length'] extends 0 ? true : false
{
    type _ = IsEmptyTuple<[]> // true
    type __ = IsEmptyTuple<[1]> // false
    type ___ = IsEmptyTuple<number[]> // false
}

/**
 * If Cache is empty return Prop without dot,
 * to avoid ".user"
 */
type HandleDot<
    Cache extends string,
    Prop extends string | number
    > =
    Cache extends ''
    ? `${Prop}`
    : `${Cache}.${Prop}`

/**
 * Simple iteration through object properties
 */
type HandleObject<Obj, Cache extends string> = {
    [Prop in keyof Obj]:
    // concat previous Cacha and Prop
    | HandleDot<Cache, Prop & string>
    // with next Cache and Prop
    | Path<Obj[Prop], HandleDot<Cache, Prop & string>>
}[keyof Obj]

type Path<Obj, Cache extends string = ''> =
    (Obj extends PropertyKey
        // return Cache
        ? Cache
        // if Obj is Array (can be array, tuple, empty tuple)
        : (Obj extends Array<unknown>
            // and is tuple
            ? (IsTuple<Obj> extends true
                // and tuple is empty
                ? (IsEmptyTuple<Obj> extends true
                    // call recursively Path with `-1` as an allowed index
                    ? Path<PropertyKey, HandleDot<Cache, -1>>
                    // if tuple is not empty we can handle it as regular object
                    : HandleObject<Obj, Cache>)
                // if Obj is regular  array call Path with union of all elements
                : Path<Obj[number], HandleDot<Cache, number>>)
            // if Obj is neither Array nor Tuple nor Primitive - treat is as object    
            : HandleObject<Obj, Cache>)
    )

type WithDot<T extends string> = T extends `${string}.${string}` ? T : never


// "user" | "user.arr" | `user.arr.${number}`
type Test = WithDot<Extract<Path<Structure>, string>>



type Acc = Record<string, any>

type ReducerCallback<Accumulator extends Acc, El extends string> =
    El extends keyof Accumulator ? Accumulator[El] : El extends '-1' ? never : Accumulator

type Reducer<
    Keys extends string,
    Accumulator extends Acc = {}
    > =
    // Key destructure
    Keys extends `${infer Prop}.${infer Rest}`
    // call Reducer with callback, just like in JS
    ? Reducer<Rest, ReducerCallback<Accumulator, Prop>>
    // this is the last part of path because no dot
    : Keys extends `${infer Last}`
    // call reducer with last part
    ? ReducerCallback<Accumulator, Last>
    : never

{
    type _ = Reducer<'user.arr', Structure> // []
    type __ = Reducer<'user', Structure> // { arr: [] }
}

type BlackMagic<T> = T & {
    [Prop in WithDot<Extract<Path<T>, string>>]: Reducer<Prop, T>
}

type Result = BlackMagic<Structure>

Playground

This implementation is worth considering

9
  • Thanks to provide an alternative (also brilliant) way. I did some test with it, found it fails to detect array type. Is it possible to work with it?
    – Val
    Sep 10, 2021 at 8:49
  • I tried the playground, I found a few not working caveats / edge cases in the code. for example, it won't work if I change "bar.baz" to array type (and also bar.gaz.zaz). And to ExludeHighLevel from KeysUnion, Structure & right after looks not so good to me (and may cause more exceptions). Any chance to improve it?
    – Val
    Sep 13, 2021 at 4:15
  • 1
    I'm not sure how do you want to handle arrays. Assume you have an empty array. Do you want me to generate: baz.0? Regarding the { zaz: 2 }[]. Neither I nor typescript is unable to figure out what indexes are allowed and what are disallowed. How many indexes should I generate in case of array? From the other hand, it is much easier to work with tuples because TS is able to figure out the length Sep 13, 2021 at 8:18
  • 1
    You're right, I should list down the use cases that I need, or there could be too many edge cases there. The handle of array / index signature by $number is good enough. I've added a list of use cases into the question, please have a look with it =) Thanks for the answer and patient explanation, I'll read into it after work (its complexity explode my head, oops)
    – Val
    Sep 14, 2021 at 4:19
  • 1
    @captain-yossarian This is absolutely wonderful. I noticed that the final BlackMagic type has a typo, it is hard-coded to Structure :) Nov 29, 2021 at 21:09

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.