CodeSamplez.com

Programming, Web development, Cloud Technologies

  • Facebook
  • Google+
  • RSS
  • Twitter
  • Home
  • Featured
    • C# Tutorials
      • LinQ Tutorials
      • Facebook C# API Tutorials
    • PHP Tutorials
      • CodeIgniter Tutorials
    • Amazon AWS Tutorials
  • Categories
    • Programming
    • Development
    • Database
    • Web Server
    • Source Control
    • Management
    • Project
  • About
  • Write
  • Contact
Home Programming Multiple Objects From Aamazon S3 With PHP SDK

Multiple Objects From Aamazon S3 With PHP SDK

Rana Ahsan February 23, 2014 1 Comment


 Multiple Objects From Aamazon S3 With PHP SDK    

Recently I was working on a task, that requires retrieving multiple media files/objects from amazon s3 service, process them to create preview/thumbnail versions and show them on a web page. It is pretty straightforward tasks except the performance issue. What is gonna happen when there are several files(as the number of files is dynamic) and retrieving them one after another would take a long time, even before any kind of processing.

To solve this, we came up with the idea to make the requests in parallel. Which will reduce the time to a small fraction(time to retrieve the longest file). If you don’t know, the official amazon php sdk utilizes guzzle library to perform the http requests. Luckily, guzzle also started support performing parallel requests from their version 2.0. Which makes the job a lot easier for us. Here I am sharing the code I came up with to achieve this goal. If you are looking for solutions to similar issues as mine, you can also might get help from this.

Lets see the main class code first(gist link):

To use this, you can use code as below:

s3 = new \S3\MyS3Client();
$configs = array();
$configs[] = array(
                'Bucket' => "{test-bucket-name}",
                'Key'    => "{test-key-for-object}",
                'saveAs' =>  "{local-path-to-save}"
            );
//add more as you need similar to above
//$configs[] = ....

//retrieve all
\S3\MyS3Client::getObjects($configs, $s3);

See, its as simple as that. Let me explain this small code.

the main source code actually extends the original S3Client class provided by amazon php sdk. So, you can use it completely as if you we re using the original class. Or, if adding it into an existing project, just change the instantiation of the s3 client variable with this new class.

Now, construct the config as if you are constructing to retrieve single object and merge them into another array. that’s it. Though for single object retrieval, ‘saveAs’ is optional parameter and it returns the content, here is our case, we must need to pass this so that all objects get saved as per given path/name.

original S3Client class is made up in a way that, all methods are mapped into aws SDK REST API commands. Additional methods are static. Thus, we made this new method static as well. However, if you have a good suggestion how we can get it working in normal method instead of static one, I am open to it.

Hope this small tutorial to retrieve multiple objects from S3 will help you in some extent to optimize your web app and perform better. Happy programming 🙂

Related

Filed Under: Programming Tagged With: aws, php

About Rana Ahsan

Rana is a passionate software engineer/Technology Enthusiast.
Github: ranacseruet

Trackbacks

  1. PHP HTTP Request With Guzzle - CodeSamplez says:
    June 11, 2014 at 7:55 pm

    […] function that might seem easier for you. But, in terms of performance, cURL still should be your primary choice. So, instead of digging into complexity of cURL library, this library will give you easy to understand API interface. Also, with guzzle, you will able to take the full power of curl to perform multiple parallel requests simultaneously as I described on my another article about retrieving multiple objects from AWS S3 in parallel. […]

    Reply

Leave a Reply Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Email Subscription

Never miss any programming tutorial again.

Popular Tutorials

  • PHP HTML5 Video Streaming Tutorial
  • How To Work With JSON In Node.js / JavaScript
  • Generate HTTP Requests using c#
  • How To Work With C# Serial Port Communication
  • Facebook C# API Tutorials
  • How To Work With CodeIgniter Pagination
  • LinQ To SQL Database Update Operations In C#
  • Get Facebook C# Api Access Token
  • How To Work With Multithreaded Programming In C#.NET Application
  • Getting Started With HTML5 Web Speech API

Recent Tutorials

  • Building Auth With JWT – Part 1
  • Document Your REST API Like A Pro
  • Understanding Golang Error Handling
  • Web Application Case Studies You Must Read
  • Getting Started With Golang Unit Testing
  • Getting Started With Big Data Analytics Pipeline
  • NodeJS Tips And Tricks For Beginners
  • Apple Push Notification Backend In NodeJS
  • Web Based Universal Language Translator, Voice/Text Messaging App
  • How To Dockerize A Multi-Container App From Scratch

Recent Comments

  • intolap on PHP HTML5 Video Streaming Tutorial
  • manishpanchal on PHP HTML5 Video Streaming Tutorial
  • Rana Ghosh on PHP HTML5 Video Streaming Tutorial
  • ld13 on Pipe Email To PHP And Parse Content
  • Daniel on PHP HTML5 Video Streaming Tutorial

Archives

Resources

  • CodeSamplez.com Demo

Tags

.net apache api audio aws c# cache cloud server codeigniter deployment doctrine facebook git github golang htaccess html5 http image java javascript linq mysql nodejs oop performance php phpmyadmin plugin process python regular expression scalability server smarty ssh tfs thread tips ubuntu unit-test utility web application wordpress wpf

Copyright © 2010 - 2021 · CodeSamplez.com ·

Copyright © 2021 · Streamline Pro Theme on Genesis Framework · WordPress · Log in