The official NestJS documentation explains very clearly how to handle multiple database connections in a static configuration.
One of our clients has a different use-case and wants the application to be able to connect to many databases, each one dedicated to a country. He wants to add or remove a database dynamically without changing the codebase.
Here are the main differences between these two models :
| Static | Dynamic | |
| Database structure | Same or different | Has to be the same |
| Add / remove database | Codebase change | Configuration change and application restart |
How to setup dynamic configuration
Here is the .env file of what we want to achieve :
DATABASE_SYSTEM_IDS=FR,DE,GB
DB_FR_TYPE=postgres
DB_FR_HOST=localhost
DB_FR_PORT=5432
DB_FR_USERNAME=fr
DB_FR_PASSWORD=fr
DB_FR_DATABASE=fr
DB_FR_SYNCHRONIZE=true
DB_DE_TYPE=postgres
DB_DE_HOST=localhost
DB_DE_PORT=5433
DB_DE_USERNAME=de
DB_DE_PASSWORD=de
DB_DE_DATABASE=de
DB_DE_SYNCHRONIZE=true
DB_GB_TYPE=postgres
DB_GB_HOST=localhost
DB_GB_PORT=5434
DB_GB_USERNAME=gb
DB_GB_PASSWORD=gb
DB_GB_DATABASE=gb
DB_GB_SYNCHRONIZE=true DATABASE_SYSTEM_IDS is a coma separated list of arbitrary defined system identifiers to all the databases we want to connect to. For each one of them, we have DB_xx_* variables where xx is the database system identifier.
Below is the configuration class which can handle this kind of configuration (file src/common/config/orm.config.ts) :
import { registerAs } from '@nestjs/config';
import * as dotenv from 'dotenv';
dotenv.config(); // used to get process.env access prior to AppModule instanciation
export const getDatabaseSystemIds = (): string[] => {
return process.env.DATABASE_SYSTEM_IDS.split(',');
};
export default registerAs('orm', () => {
const config = {};
getDatabaseSystemIds().forEach((systemId) => {
config[systemId] = {
type: process.env[`DB_${systemId}_TYPE`],
host: process.env[`DB_${systemId}_HOST`],
port: parseInt(process.env[`DB_${systemId}_PORT`]),
username: process.env[`DB_${systemId}_USERNAME`],
password: process.env[`DB_${systemId}_PASSWORD`],
database: process.env[`DB_${systemId}_DATABASE`],
synchronize: process.env[`DB_${systemId}_SYNCHRONIZE`] === 'true',
entities: [`${__dirname}/.. /.. /**/*.entity{.ts,.js}`],
};
});
return config;
}); Here is how we generate dynamically a TypeORM configuration per database (file src/home/jnesis/www/wp.module.ts) :
import { Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';
import { TypeOrmModule } from '@nestjs/typeorm';
import { CarManufacturersModule } from './car-manufacturers/car-manufacturers.module';
import ormConfig, { getDatabaseSystemIds } from './common/config/orm.config';
// database connection for each system id
const databasesConfig = getDatabaseSystemIds().map((systemId) => {
return TypeOrmModule.forRootAsync({
name: `database-${systemId}`,
imports: [ConfigModule.forFeature(ormConfig)],
useFactory: (config: ConfigService) => config.get(`orm.${systemId}`),
inject: [ConfigService],
});
});
@Module({
imports: [
ConfigModule.forRoot({
isGlobal: true,
}),
...databasesConfig,
CarManufacturersModule,
],
controllers: [],
providers: [],
})
export class AppModule {} Note each one of these TypeORM configurations is assigned with the name database-xx where xx is the database system identifier.
How to setup dynamic dependency injection
Now our configuration is dynamic, we need to be able to connect to a specific database through its system identifier. Standard static dependency injection through (@InjectEntityManager , @InjectRepository, etc…) cannot be used as it requires to know the database we want to connect to at compile time.
To be able to dynamically select a database on runtime, we need to use the module reference feature from NestJS. This way, we can retrieve an EntityManager from its TypeORM configuration name database-xx (see our loadEntityManager method in file src/car-manufacturers/car-manufacturers.service.ts) :
import { Injectable } from '@nestjs/common';
import { ModuleRef } from '@nestjs/core';
import { getEntityManagerToken } from '@nestjs/typeorm';
import { EntityManager } from 'typeorm';
import { CarManufacturer } from './car-manufacturer.entity';
@Injectable()
export class CarManufacturersService {
constructor(private moduleRef: ModuleRef) {}
private async loadEntityManager(systemId: string): Promise<EntityManager> {
return this.moduleRef.get(getEntityManagerToken(`database-${systemId}`), {
strict: false,
});
}
async findAll(countryCode: string): Promise<CarManufacturer[]> {
const entityManager = await this.loadEntityManager(countryCode);
if (!entityManager) {
return [];
}
return entityManager.find(CarManufacturer);
}
} Demo project
You’ll find the complete working example project here in Github.
Same database structure limitation
Actually we could have different structure in dynamic databases. It requires to define the shape of each database specifying the set of entities it uses.
As an example we could add a parameter to the .env file :
DB_FR_ENTITIES=structureA
DB_DE_ENTITIES=structureB
DB_GB_ENTITIES=structureA And then dynamically load entities depending using the structure name as a suffix in file src/common/config/orm.config.ts :
entities: [`${__dirname}/.. /.. /**/*.${process.env[`DB_${systemId}_ENTITIES`]}.entity{.ts,.js}`], You’ll have to define entities for each database structure, and they can be completely different :
- StructureA: car-manufacturer.structureA.entity.ts
- StructureB: brand.structureB.entity.ts, address.structureB.entity.ts
Obviously the database queries will also be different from one database structure to another, so it will require additional conditions and branching in the business code.
2 Responses
Hi,
excellent content. Could you please provide a full CRUD Operations of this?
I’m having trouble getting the CRUD Operations working with this method since I couldn’t find any documentation of it.
Hi,
Thank you for your feedback !
Once you get that EntityManager instance (as shown in CarManufacturersService) you can do any regular CRUD and other database operations you want.
Have a look at this :
– EntityManager introduction: https://typeorm.io/working-with-entity-manager
– EntityManager API: https://typeorm.io/entity-manager-api#entitymanager-api
Please note you could get a Repository instance instead of an EntityManager replacing “getEntityManagerToken(…)” method with “getRepositoryToken(…)”.