Milesoft Storage Extension (storage-ext)

1. Overview & Purpose

The Storage Extension (storage-ext) is a unified, highly optimized, and robust developer utility designed to provide a secure and standardized facade around Google Cloud Storage (GCS). It simplifies object storage interactions within the Milesoft ecosystem by offering a simple API to read, write, and delete files, as well as generate time-limited signed URLs for secure direct-to-cloud client interactions.

Key Benefits & Core Capabilities:

• Standardized Facade: Provides a clean, type-safe API for reading and writing string or byte payloads, shielding downstream developers from raw Google Cloud API boilerplate.
• Time-Limited Signed URLs: Natively supports secure URL generation for both client download (GET) and direct upload (PUT) workflows without exposing private cloud credentials.
• Seamless Testing & Mocking: Exposes package-private constructors that facilitate 100% test coverage using standard mock frameworks (e.g. Mockito) in downstream microservices.
• Consistent Validation: Integrates built-in argument and null-pointer checks to catch configuration and runtime errors early, before making expensive cloud network calls.

---

2. Installation & Dependency Configuration

The Storage Extension is hosted securely inside our Google Cloud Artifact Registry repository.

To configure your application to resolve and pull this dependency:

2.1 Gradle Setup (build.gradle)

Apply the Google Cloud Artifact Registry plugin and configure the Maven repository block:

plugins {
    id "com.google.cloud.artifactregistry.gradle-plugin" version "2.2.5"
    id "java"
}

repositories {
    mavenLocal()
    mavenCentral()
    maven {
        url "artifactregistry://us-west2-maven.pkg.dev/milesoft-repo/repo-maven"
    }
}

dependencies {
    // Milesoft Commons foundation (required by storage-ext)
    implementation "io.milesoft:commons:2.0.1"
    
    // Storage Integration Extension
    implementation "io.milesoft:storage-ext:2.0.1"
}

---

3. Configuring Storage in Spring Boot

Downstream Spring Boot microservices and applications utilize a standard configuration pattern to initialize and inject the StorageHelper as a Spring-managed bean.

3.1 Define Application Properties

Configure your target GCS bucket name in your properties configuration (e.g., application.yml or application.properties):

storage:
  bucket: "milesoft-production-assets"

3.2 Defining Storage Configuration Bean

To enable the StorageHelper bean inside your application context, declare it in a standard configuration class:

import io.milesoft.storage.utils.StorageHelper;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class StorageConfig {

    @Bean
    public StorageHelper storageHelper() {
        return new StorageHelper();
    }
}

---

4. Key Components

4.1 StorageHelper

The single, thread-safe orchestrator utility provided by this extension. It manages connections to Google Cloud Storage and acts as the entry point for all storage transactions, encapsulating standard operations on GCS Storage and GCS Blob objects.

---

5. API Reference Guide

5.1 StorageHelper Class

The class exposes the following public methods for handling storage operations.

Constructors:

• public StorageHelper() Creates a new helper instance, resolving GCS credentials and configurations automatically via the default environment/IAM settings (StorageOptions.getDefaultInstance().getService()).

Core Methods:

Method Signature	Return Type	Description
getDownloadSignedUrl(String bucketName, String fileName)	String	Generates a time-limited GET signed URL valid for 1 hour. Useful for secure, direct file downloads or media rendering on frontends.
getUploadSignedUrl(String bucketName, String fileName, String contentType)	String	Generates a time-limited PUT signed URL valid for 1 hour with a specified contentType. Allows clients to upload large payloads directly to GCS without hitting the application backend.
write(String bucketName, String fileName, String content)	void	Writes string content (converted as UTF-8) to the target GCS bucket and file path. Throws an IllegalArgumentException if content or paths are blank.
write(String bucketName, String fileName, byte[] content)	void	Writes raw binary byte content to the target GCS bucket and file path. Throws appropriate exceptions if the byte array is null or empty.
readStr(String bucketName, String fileName)	Optional<String>	Reads the file contents from GCS and decodes them as a UTF-8 string. Returns Optional.empty() if the file does not exist, rather than throwing.
readBytes(String bucketName, String fileName)	Optional<byte[]>	Reads raw binary bytes from the target GCS bucket and file path. Returns Optional.empty() if the file does not exist.
delete(String bucketName, String fileName)	void	Deletes the specified object/file from GCS. No-op if the file is not found.

---

6. Practical Real-World Code Examples

These integration examples represent standard, battle-tested patterns for reading/writing files and handling signed URL workflows across the Milesoft codebase.

Example A: Injecting and Using StorageHelper in a Document Service

This example demonstrates a service that uploads metadata to a database (e.g. Firestore) while storing the actual file content on Google Cloud Storage using StorageHelper.

import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkArgument;

import io.milesoft.storage.utils.StorageHelper;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import java.util.Optional;

@Service
public class DocumentService {

    private final StorageHelper storage;
    private final String bucket;

    @Autowired
    public DocumentService(StorageHelper storage, @Value("${storage.bucket}") String bucket) {
        checkArgument(StringUtils.isNotBlank(bucket), "bucket must not be blank");
        this.storage = checkNotNull(storage, "storage helper must not be null");
        this.bucket = bucket;
    }

    /**
     * Uploads file content to the configured GCS bucket using the document ID as key.
     */
    public void uploadDocument(String documentId, byte[] fileContent) {
        checkArgument(StringUtils.isNotBlank(documentId), "documentId must not be blank");
        checkNotNull(fileContent, "fileContent must not be null");

        // Write raw bytes directly to storage
        storage.write(bucket, documentId, fileContent);
    }

    /**
     * Downloads file content from the configured GCS bucket.
     */
    public byte[] downloadDocument(String documentId) {
        checkArgument(StringUtils.isNotBlank(documentId), "documentId must not be blank");

        return storage.readBytes(bucket, documentId)
            .orElseThrow(() -> new RuntimeException("Document not found in storage: " + documentId));
    }
}

Example B: Secure Direct-to-Cloud Upload & Download Workflow

Directly proxying binary transfers through an application server can cause memory pressure and latency. A better approach is using Signed URLs to allow the frontend client to securely interact directly with Google Cloud Storage.

import io.milesoft.storage.utils.StorageHelper;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;

@Service
public class SignedUrlService {

    private final StorageHelper storage;
    private final String bucket;

    public SignedUrlService(StorageHelper storage, @Value("${storage.bucket}") String bucket) {
        this.storage = storage;
        this.bucket = bucket;
    }

    /**
     * Generates a secure, temporary PUT URL. The frontend can perform an HTTP PUT request 
     * directly to this URL with the file payload.
     */
    public String generateUploadUrl(String fileName, String contentType) {
        return storage.getUploadSignedUrl(bucket, fileName, contentType);
    }

    /**
     * Generates a temporary GET URL. The frontend can render or download this file directly 
     * in the browser for up to 1 hour.
     */
    public String generateDownloadUrl(String fileName) {
        return storage.getDownloadSignedUrl(bucket, fileName);
    }
}

Example C: Unit Testing Services with Mocked StorageHelper

This standard pattern shows how to mock StorageHelper when testing your services, avoiding actual external network calls and verifying that the correct integration methods are invoked.

import static org.mockito.Mockito.*;
import static org.junit.Assert.*;

import io.milesoft.storage.utils.StorageHelper;
import java.nio.charset.StandardCharsets;
import java.util.Optional;
import org.junit.Before;
import org.junit.Test;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;

public class DocumentServiceTest {

    @Mock
    private StorageHelper storageHelper;

    private DocumentService documentService;
    private final String testBucket = "test-bucket";

    @Before
    public void setUp() {
        MockitoAnnotations.openMocks(this);
        documentService = new DocumentService(storageHelper, testBucket);
    }

    @Test
    public void testUploadDocument() {
        String docId = "doc-123";
        byte[] content = "Hello World".getBytes(StandardCharsets.UTF_8);

        documentService.uploadDocument(docId, content);

        // Verify write was called on GCS
        verify(storageHelper, times(1)).write(testBucket, docId, content);
    }

    @Test
    public void testDownloadDocument_Success() {
        String docId = "doc-123";
        byte[] expectedContent = "Hello World".getBytes(StandardCharsets.UTF_8);

        when(storageHelper.readBytes(testBucket, docId)).thenReturn(Optional.of(expectedContent));

        byte[] actualContent = documentService.downloadDocument(docId);

        assertArrayEquals(expectedContent, actualContent);
        verify(storageHelper, times(1)).readBytes(testBucket, docId);
    }

    @Test(expected = RuntimeException.class)
    public void testDownloadDocument_NotFound() {
        String docId = "doc-999";

        when(storageHelper.readBytes(testBucket, docId)).thenReturn(Optional.empty());

        documentService.downloadDocument(docId);
    }
}