# amazon-dsstne

**Repository Path**: cyhjb/amazon-dsstne

## Basic Information

- **Project Name**: amazon-dsstne
- **Description**: DSSTNE（读作 destiny）是亚马逊开源深度学习工具。亚马逊开发者认为人类对于深度学习的研究将不止于语言理解及视觉识别，智能搜索和推荐将得到更广泛的研究和讨论。他们希望和世界上的顶级研究者们合作，一起提升这方面的研究进程。
这套深度学习系统目前还有很大的局限性，比如不支持卷积神经网络，这导致了它不能用于图像识别，对于递归神经网络也支持不佳。然而，这套软件最大的亮点在于能同时支持两个图形处理器（GPU）参与运算，这是其他深度学习工具做不到的。
- **Primary Language**: Unknown
- **License**: Apache-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 0
- **Created**: 2020-12-13
- **Last Updated**: 2021-03-02

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README



# Amazon DSSTNE: Deep Scalable Sparse Tensor Network Engine

DSSTNE (pronounced "Destiny") is an open source software library for training and deploying recommendation
models with sparse inputs, fully connected hidden layers, and sparse outputs. Models with weight matrices
that are too large for a single GPU can still be trained on a single host. DSSTNE has been used at Amazon
to generate personalized product recommendations for our customers at Amazon's scale. It is designed for
production deployment of real-world applications which need to emphasize speed and scale over experimental 
flexibility.

DSSTNE was built with a number of features for production recommendation workloads:

* **Multi-GPU Scale**: Training and prediction
both scale out to use multiple GPUs, spreading out computation
and storage in a model-parallel fashion for each layer.
* **Large Layers**: Model-parallel scaling enables larger networks than
are possible with a single GPU.
* **Sparse Data**: DSSTNE is optimized for fast performance on sparse datasets, common in recommendation 
problems. Custom GPU kernels perform sparse computation on the GPU, without filling in lots of zeroes.

## Benchmarks
* scottlegrand@ reported [near-linear scaling with multiple GPUs] on the MovieLens recommendation problem 
(https://medium.com/@scottlegrand/first-dsstne-benchmarks-tldr-almost-15x-faster-than-tensorflow-393dbeb80c0f#.ghe74fu1q)
* Directions on how to run a benchmark can be found in [here](benchmarks/Benchmark.md)

## Scaling up
* [Using Spark in AWS EMR and Dockers in AWS ECS ](http://blogs.aws.amazon.com/bigdata/post/TxGEL8IJ0CAXTK/Generating-Recommendations-at-Amazon-Scale-with-Apache-Spark-and-Amazon-DSSTNE)
    

## License
[License](LICENSE)


 
 

## Setup
* Follow [Setup](docs/getting_started/setup.md) for step by step instructions on installing and setting up DSSTNE

## User Guide
* Check [User Guide](docs/getting_started/userguide.md) for detailed information about the features in DSSTNE

## Examples
* Check [Examples](docs/getting_started/examples.md) to start trying your first Neural Network Modeling using DSSTNE

## Q&A
[FAQ](FAQ.md)