My aim is to streamline the processing of different types of batches using one program by specifying the batch type in the .env file.
The reason for wanting to process all batches with a single program is to avoid configuring a separate Continuous Deployment (CD) pipeline for each batch type.
Even if the program uses the same image, I desired to execute different operations based on the batch type, which is why I utilized .env. This ensures that only one batch type is processed each time the program runs.
Initially, when there were only a few batch types, I wrote the functions directly.
function oldOne(){
const batchType = process.env.BATCH_TYPE
if(batchType === EnumBatchType.AD_KEYWORD_INPUT){
// perform actions specific to ad keyword input
}else if(batchType === EnumBatchType.AD_KEYWORD_OUTPUT){
// perform actions specific to ad keyword output
}
...
process.exit(0)
}
As the number of messages to handle exceeded 10, I opted to create a class that inherits an interface capable of managing those messages.
This approach proved to be quite satisfying.
export interface Batchable{
doBatch():Promise<void>; // every batch class includes this function
}
------ InputBatch, OutputBatch, RefineBatch, and so on...
export abstract class InputBatch<T> implements Batchable {
abstract getDataFromKurve(): Promise<T[]>
abstract getBodyFromData(datas: T[]): Promise<NexusTagDto>
abstract updateNexusTag(body: NexusTagDto): Promise<NexusResponseDto>
async doBatch(): Promise<void> {
const datas = await this.getDataFromKurve()
const body = await this.getBodyFromData(datas)
const res = await this.updateNexusTag(body)
unifiedLog('batch-finish', JSON.stringify(res), true)
}
}
------
export class Batch1 extends InputBatch<Something> {...}
export class Batch2 extends InputBatch<Something> {...}
export class Batch3 extends OutputBatch<Something> {...}
export class Batch4 extends RefineBatch<Something> {...}
export class Batch5 extends InputBatch<Something> {...}
export class Batch6 extends OutputBatch<Something> {...}
export class Batch7 extends InputBatch<Something> {...}
export class Batch8 extends InputBatch<Something> {...}
Although each batch has a descriptive name, for security purposes, they are labeled as Batch1, Batch2, etc.
Excluding the switch statements and individual batch codes, the rest of the code is presented below.
export async function batchStart() {
const batchType = process.env.BATCH_TYPE
unifiedLog('batch_start', process.env.TARGET_SCHEMA)
const client = getBatchClient(batchType as EnumBatchType)
if (client) {
await client.doBatch()
process.exit(0)
}
}
As the number of batch types surpassed 60, the function getBatchClient with just switch-case statements escalated to over 200 lines of code.
Due to company guidelines mandating alphabetical sorting of switch statements, navigating the code has become cumbersome.
Considering the likelihood of an increase in batch types, the long switch statements are no longer feasible.
Contemplating the use of a map with key-value pairs for message-client handling, it becomes apparent that using a single message handler and terminating the program is not optimal.
It seems wasteful to create numerous classes when only one will be utilized.
function useMapCase(batchType: EnumBatchType): Batchable {
const map = new Map<EnumBatchType, Batchable>([
[EnumBatchType.BATCH_TYPE1, new Batch1()],
[EnumBatchType.BATCH_TYPE2, new Batch2()],
[EnumBatchType.BATCH_TYPE3, new Batch3()],
[EnumBatchType.BATCH_TYPE4, new Batch4()],
[EnumBatchType.BATCH_TYPE5, new Batch5()],
[EnumBatchType.BATCH_TYPE6, new Batch6()]
])
return map.get(batchType)
}
If the number of batches exceeds 200, the map constructor alone could stretch over 200 lines.
Unfortunately, without the authority to alter the layout for gradual setup, the task of managing such large maps becomes overwhelming.
Exploring alternative methods presented in other articles, one such approach is demonstrated below:
function otherWays(batchType: SomeObject): Batchable {
const { batchType, level1, level2 } = batchType
try {
const batchTypeMap = new Map<string, BatchSelector>([
['input', new InputBatchSelector()],
['output', new OutputBatchSelector()],
['refine', new RefineBatchSelector()]
])
const level1Map = batchTypeMap.get(batchType)
const level2Map = level1Map.get(level1)
return level2Map.get(level2)
} catch (e) {
unifiedError('invaild_type', batchType)
new SlackClient('Init Batch').sendErrorMessage({
message: `${JSON.stringify(batchType)} is invalid type (from ${process.env.JOBTAG})`,
target: EnumNotiTarget.BATCHCODE
})
return null
}
}
Though this approach divides the process into segments and avoids the need to create entire classes, it may involve multiple object instantiations.
For a streamlined and efficient solution, there might need to be an established pattern in the batch type, which unfortunately is not the case here.
Amidst these challenges, the dilemma remains whether to handle hundreds of cases individually or to utilize a map despite the inefficient memory utilization.