Release Summary - Sep 19, 2024 (24.18)

ENG-28862

Recommend:

Failsafe Checks and Internal Timeouts for Recs API Processing

To improve platform performance, internal timeouts and failsafe checks have been added to Recs API processing. If a request takes more than 3 to 4 seconds, it will be automatically timed out to prevent performance issues caused by misconfigured placements.

Proactive monitoring has been integrated with Datadog to track these events, logging key details like Site ID to help quickly identify and resolve potential issues before they affect overall system stability.

ENG-28408

Data Engineering:

TS Migration to Azure Snowflake: Scheduled Job for Data Transfer to Azure Blob Storage

We have introduced a scheduled job to ensure that rollup data is regularly updated to Azure Blob Storage for all reports related to ThoughtSpot's migration to Azure Snowflake. This job runs daily to maintain up-to-date reporting, with monitoring in place to ensure data transfer continuity and to alert in case of failures.

ENG-28530

Data Engineering:

TS Migration to Azure Snowflake: Scheduled Job for Dimension Tables Data Transfer

A scheduled job has been implemented to ensure that dimension table data is updated daily to Azure Blob Storage as part of the ThoughtSpot migration to Azure Snowflake. Monitoring has been set up to track data transfers and notify in case of any failures, ensuring seamless and up-to-date reporting.

ENG-29109

Direct Streaming Ingest for Query Vector Data

The Query Vector job has been enhanced to write data directly to streaming ingest, eliminating the need for an additional query ingest job. Previously, query vectors were stored in enrichment for further processing, but with the transition to the metastore collection, this intermediate step is no longer necessary. Now, the Query Vector job directly ingests both query and embedding into the live snapshot, streamlining the process and removing the dependency on the query ingest job.

ENG-29020

Find Data Publisher: Allow Live Dataset Ingestion into Enrichment

The Find Data Publisher now supports live dataset ingestion into enrichment without creating new datasets if one already exists. After successfully running catalog-vector and query-vector jobs, the system ensures that enrichment ingestion occurs within the live dataset.

ENG-29014

Find:

RRServer Find Cluster Service Enhancements

Improvements have been made to RRServer Find Cluster Service for better error handling and cluster management:

• Missing Cluster Mapping: If a cluster pool is missing, RRServer will now attempt to retrieve data from a secondary cluster before throwing an error.

• Cluster Association State: RRServer will skip clusters marked as "NOT READY," ensuring only available clusters are used.

• Error Handling for Unavailable Clusters: If no clusters can handle a request, an error message will now be returned instead of an empty result.

ENG-29033

Find:

New Snapshot Type: Metastore for Query Vectors

We have introduced a new snapshot type called "metastore" to store query vectors. This new snaoshot has a new Solr collection called "metastore" which is language specific. The streaming find indexer has been updated to save query vectors to metastore collection instead of catalogtagger collection. The search service has been updated to use the metastore collection instead of the catalogtagger collection for retrieving query vectors.

Additionally, the necessary Solr configurations for metastore have been added to language service to ensure seamless integration.

PLAT-3884

Streaming Catalog:

Support for GPT-4o-Mini Integration

We have added support for the GPT-4o-Mini model by updating the LLM account configurations. New LLM accounts for specific sites have been created, allowing the use of GPT-4o-Mini under the LLM type "azureOpenAiChatMini." This enhancement enables users to define and utilize prompts for the GPT-4o-Mini model. All relevant APIs for managing these configurations and retrieving LLM types are fully operational.

ENG-28867

Enterprise Dashboard:

Fix for Segment ID Duplication in Strategy Rules

An issue where segment IDs were being duplicated each time a Strategy Rule was saved has been resolved. Previously, the repeated addition of segment IDs caused inflated values in certain rules. This fix ensures that segment IDs are no longer duplicated during the saving process.

ENG-28936

Find:

Case Sensitivity Issue in Offmenu

The case sensitivity issue with the "offMenu" feature has been fixed. Previously, exact matches were not case-insensitive as expected, causing discrepancies in search results. For example, lowercase and uppercase queries like "2H35j" and "2H35J" returned different results. This has been corrected, and offMenu now handles exact matches case-insensitively as intended.

ENG-29071

Catalog Vector Job Error

We have resolved an issue where the catalog vector job was failing due to a missing configuration in Consul. The job was producing only delta updates and encountered errors related to missing settings for hdfsBaseInputPath. After adding the necessary configuration, the catalog vector job is now functioning correctly.

ENG-28998

Dynamic Experiences:

Multiple Click Events in Click-Trigger Class

The issue where the click-trigger class was firing multiple click events (up to 6 times) from previously listened elements in client.js has been resolved. Now, when the submit button is clicked, only a single click event is triggered as expected, preventing duplicate tracking events.

ENG-29065

Recommend, rrportal:

CatalogInfo API Returning Empty Data

The CatalogInfo API previously returned empty data if one or more non-existent category IDs were included in the request. This issue has been resolved, and now the API properly returns an empty/null object for non-existing categories while providing valid data for existing categories.

PLAT-3904

Recommend:

Excessive SKU + Region Parsing in Strategy Execution Service

A performance issue related to SKU + region parsing in the Strategy Execution Service (SES) has been resolved. Previously, the SES repeatedly parsed SKU data due to a lack of caching, leading to high computation and memory allocation, especially for high-load clients like Sams Club. This has been addressed by properly wrapping SKU-related methods in ProductEntity, improving efficiency. The recsForPlacement and recsForPlacementContext API calls are now functioning as expected.

ENG-29039

Enterprise Dashboard, Social Message:

Incorrect Threshold and Message Display in Social Proof

An issue where the threshold and message values were not updating correctly in the Social Proof campaign page has been resolved. Previously, custom thresholds and messages for "Views Now" and "Purchases Today" were not reflecting properly, defaulting to incorrect values. This has been fixed, and the campaign now accurately displays the set thresholds and messages.

ENG-29025

Data Engineering:

Data Transfer to Rollup List Not Working for Visit-Dependent Rollups

An issue where data transfer to the rollup list was not correctly differentiating between visit-dependent and non-visit-dependent rollups has been resolved. Now, when directives are set for visit-dependent or non-visit-dependent rollups, the system transfers only the relevant rollup data. Additionally, blacklisted rollups are excluded, and account access keys are used for importing labels to cloud storage.

ENG-28993

Find:

ConcurrentModificationException in Client.js When Timing is Enabled

An issue causing a ConcurrentModificationException when timing is enabled in client.js has been resolved. The error occurred when parsing SKU data for Strategy Execution Service, resulting in excessive computation and multiple click events being triggered. This issue is now fixed, and the timing is set to false by default, preventing the exception from being thrown.

ENG-28960

Enterprise Dashboard, Find:

UI Label Corrections for Boost Bury Search Rule

The UI labels on the Boost Bury Search Rule have been updated. Corrections include renaming "Boost Bury Search" to "Boost/Deboost Search" across various sections and fixing minor typos such as "keywork" to "keyword" and "site-wide" to "sitewide." These changes ensure clarity and consistency in the user interface.

ENG-29125

Find:

Search Index Issues with Missing Products

The issue where numerous active products were not appearing in search results has been resolved. Products that were missing when using the clusterPool parameter have now been restored to the search index, ensuring accurate and complete search results across all relevant queries.

PLAT-3897

Streaming-recommend:

MapDB3 Error During Item Updates

The issue where MapDB3 was throwing a "recid not present" error while processing item updates has been resolved. This error occurred due to separate DB instances for reading and writing, causing synchronization issues between file channels. The fix now allows the writer to handle both reading and writing through a unified interface, ensuring proper disk synchronization and preventing the error from reoccurring.