Skip to main content
Big Data Test Infrastructure (BDTI)

BDTI platform version information

Welcome to the dedicated source for all updates and news regarding the BDTI platform. Our platform is continually evolving to meet the needs of our users, and this page serves as your official resource for all the latest information.

Current Version

Stay up-to-date with the latest release of the BDTI platform. Here, you'll find comprehensive details about the most recent version, including performance improvements and security enhancements that keep your operations smooth and secure.

 

Current version: 1.1.0 (10.04.2024)

What's New

Discover the latest features and services introduced in the BDTI platform. Our commitment to innovation ensures you have access to cutting-edge tools that enhance your workflow and productivity:

  • New catalogue service MageAI.
  • Ability to modify Keycloak's login screen realm name during deployment.
  • Ability to modify which catalogue services are hidden during deployment. By default, the deprecated ones are hidden.
  • Ability to enable IRSA on catalogue services, which allows users to specify an AWS role ARN for the workload to have only specific permissions regarding the cloud provider's services.
  • Infrastructure AWS lambda function exports the EKS cluster's logs to an S3 bucket and inputs to manipulate the important details.
  • Ability to modify infrastructure Terraform AWS RDS details such as engine version, maintenance window, etc.

Recently Removed Services

To streamline our offerings and improve overall service quality, certain features and services have been discontinued. This section provides a summary of what has been removed and how these changes might affect your usage of the platform:

  • Irrelevant documentation pages
  • Deprecated services were hidden.

Changes and Enhancements

Learn about the recent modifications and enhancements. Whether it's a minor tweak or a major overhaul, you'll find detailed descriptions of what has changed and how these improvements contribute to a better user experience.

  • Migrated multiple repositories under this mono-repo.
  • Repository pipeline structure and jobs have been updated.
  • Infrastructure cluster version has been updated to run on Kubernetes 1.26
  • Infrastructure cluster default node AMI image has been updated to latest available.
  • Infrastructure cluster log retention set to 30 days.
  • Infrastructure security group to allow SSH connectivity between workloads.
  • Infrastructure databases version (AWS RDS Postgresql) has been updated from 13.11 to 14.10.
  • Infrastructure introduction of Karpenter.
  • Catalogue service airflow image has been updated with new providers.
  • Catalogue service Jupyter image has been updated with new libraries, extensions and ability to SSH to another workload. 
  • Catalogue service Jupyter-Spark image has been updated with new libraries and extensions.
  • Catalogue service Rstudio image has been updated with libraries and multi-user functionality. 
  • Catalogue service Knime image has been updated with new tools, libraries and Knime IDE version.
  • Catalogue service docker image references have been updated to point to new private ECR registry as the public one will be deprecated.
  • Docker database initialisation image script configs have been updated to reflect the new catalogue.
  • Support layer chart Prometheus-Grafana has been updated.
  • Documentation has been updated to accommodate recent monorepo changes (including deployment guide).
  • Added password field names for Jupyter, Jupyter-Spark and Rstudio to avoid secret overwrites when the same deployment alias is used between user workload deployments.
  • OpenFaaS backend functions bug with calculation and presentation of CPU cores and milli-cores was fixed.