-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
21 lines (21 loc) · 17.8 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
<!DOCTYPE html><html lang="zh-CN"><head><meta http-equiv="content-type" content="text/html; charset=utf-8"><meta content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=0" name="viewport"><meta content="yes" name="apple-mobile-web-app-capable"><meta content="black-translucent" name="apple-mobile-web-app-status-bar-style"><meta content="telephone=no" name="format-detection"><meta name="description"><title>Tom Talk | 分享我的实践、思考、积累</title><link rel="stylesheet" type="text/css" href="//cdn.bootcss.com/normalize/5.0.0/normalize.min.css"><link rel="stylesheet" type="text/css" href="//cdn.bootcss.com/pure/0.6.2/pure-min.css"><link rel="stylesheet" type="text/css" href="//cdn.bootcss.com/pure/0.6.2/grids-responsive-min.css"><link rel="stylesheet" type="text/css" href="/css/style.css?v=0.0.0"><link rel="stylesheet" href="//cdn.bootcss.com/font-awesome/4.7.0/css/font-awesome.min.css"><script type="text/javascript" src="//cdn.bootcss.com/jquery/3.1.1/jquery.min.js"></script><link rel="Shortcut Icon" type="image/x-icon" href="/favicon.ico"><link rel="apple-touch-icon" href="/apple-touch-icon.png"><link rel="apple-touch-icon-precomposed" href="/apple-touch-icon.png"></head><body><div class="body_container"><div id="header"><div class="site-name"><h1 class="hidden">Tom Talk</h1><a id="logo" href="/.">Tom Talk</a><p class="description">分享我的实践、思考、积累</p></div><div id="nav-menu"><a href="/." class="current"><i class="fa fa-home"> 首页</i></a><a href="/archives/"><i class="fa fa-archive"> 归档</i></a><a href="/about/"><i class="fa fa-user"> 关于</i></a></div></div><div id="layout" class="pure-g"><div class="pure-u-1 pure-u-md-3-4"><div class="content_container"><div class="post"><h2 class="post-title"><a href="/2017/04/18/copyset-replication.html">副本放置&Copyset Replication</a></h2><div class="post-meta">2017-04-18</div><a data-thread-key="/2017/04/18/copyset-replication.html" href="/2017/04/18/copyset-replication.html#comments" class="ds-thread-count"></a><div class="post-content"><p>在<a href="https://www.nostalk.info/2017/01/24/storage-durablity.html" target="_blank" rel="external">分布式存储系统</a> 中说明了，在一定情况下，copyset的数量不是越多越好，在恢复时间确定的情况下，找到合适的copyset的数量可以降低数据丢失的概率。</p></div><p class="readmore"><a href="/2017/04/18/copyset-replication.html">阅读更多</a></p></div><div class="post"><h2 class="post-title"><a href="/2017/03/28/a-finding-in-perftest.html">数据存储中Zipf分布</a></h2><div class="post-meta">2017-03-28</div><a data-thread-key="/2017/03/28/a-finding-in-perftest.html" href="/2017/03/28/a-finding-in-perftest.html#comments" class="ds-thread-count"></a><div class="post-content"><p>最近团队在对存储系统做一些性能测试，期间遇到了不少问题，测试过程中得出的结果也没有很好的数据支撑，所以尝试了非常多的方法来对性能问题进行定位。</p>
<p>小王童鞋是挺厉害的，使用了非常多的工具进行性能问题定位，包括iosnoop对IO请求进行跟踪、iostat进行磁盘状态记录、go-pprof从runtime层面收集性能profile数据、使用go-torch对profile生成直观的火焰图、使用trace2heatmap对延迟数据生产热力图 等等。</p></div><p class="readmore"><a href="/2017/03/28/a-finding-in-perftest.html">阅读更多</a></p></div><div class="post"><h2 class="post-title"><a href="/2017/03/24/a-hdd-disk-test.html">硬盘性能简测</a></h2><div class="post-meta">2017-03-24</div><a data-thread-key="/2017/03/24/a-hdd-disk-test.html" href="/2017/03/24/a-hdd-disk-test.html#comments" class="ds-thread-count"></a><div class="post-content"><p>各种存储系统，数据库、文件系统，在性能上无不都在与磁盘做斗争。希望能够尽量发挥系统有限的资源，提供最大化的性能。其中涉及到的技术包括</p>
<ul>
<li>page cache</li>
<li>write buffer</li>
<li>raid卡缓存</li></div><p class="readmore"><a href="/2017/03/24/a-hdd-disk-test.html">阅读更多</a></p></div><div class="post"><h2 class="post-title"><a href="/2017/03/17/storage-physical-topoloy.html">大规模存储物理架构</a></h2><div class="post-meta">2017-03-17</div><a data-thread-key="/2017/03/17/storage-physical-topoloy.html" href="/2017/03/17/storage-physical-topoloy.html#comments" class="ds-thread-count"></a><div class="post-content"><p>大规模对象存储究竟应该采用什么样子的存储物理架构?如何规划物理机架部署?如何进行网络部署?通过学习一些经典系统如GFS、HDFS，并且结合自身在这方面的经验和需求进行了一些总结。</p>
<p>考虑的因素包括</p>
<ul>
<li>故障域</li>
<li>恢复域</li></div><p class="readmore"><a href="/2017/03/17/storage-physical-topoloy.html">阅读更多</a></p></div><div class="post"><h2 class="post-title"><a href="/2017/02/19/storage-durablity-design-pattern.html">分布式存储系统可靠性-设计模式</a></h2><div class="post-meta">2017-02-19</div><a data-thread-key="/2017/02/19/storage-durablity-design-pattern.html" href="/2017/02/19/storage-durablity-design-pattern.html#comments" class="ds-thread-count"></a><div class="post-content"><p>从上文<a href="https://work-jlsun.github.io/2017/01/24/storage-durablity.html">分布式存储系统可靠性-系统估算示例</a> 中详细分析了系统可靠性量化的估算手段，并且给出了示例代码，代码的主要输入参数为如下所示。</p>
<figure class="highlight excel"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div></pre></td><td class="code"><pre><div class="line">LoseRate(S, <span class="built_in">N</span>, RepNum, <span class="built_in">T</span>, AFR)</div><div class="line"><span class="built_in">N</span>：系统中磁盘的数量（包括磁盘的容量信息）</div><div class="line">S：系统Copyset的数量</div><div class="line">RepNum：存储的备份数量</div><div class="line"><span class="built_in">T</span>：坏盘情况下的恢复时间</div><div class="line">AFR：磁盘的年度故障率</div></pre></td></tr></table></figure></div><p class="readmore"><a href="/2017/02/19/storage-durablity-design-pattern.html">阅读更多</a></p></div><div class="post"><h2 class="post-title"><a href="/2017/02/18/storage-durablity-2.html">分布式存储系统可靠性-系统估算示例</a></h2><div class="post-meta">2017-02-18</div><a data-thread-key="/2017/02/18/storage-durablity-2.html" href="/2017/02/18/storage-durablity-2.html#comments" class="ds-thread-count"></a><div class="post-content"><h4 id="1-估算示例"><a href="#1-估算示例" class="headerlink" title="1 估算示例"></a>1 估算示例</h4><p>上文<a href="https://work-jlsun.github.io/2017/01/24/storage-durablity.html">分布式存储系统可靠性-如何估算</a>中，我们提供了一些基本的估算的方法。接下来我们提供一个具体的估算的示例子。</p></div><p class="readmore"><a href="/2017/02/18/storage-durablity-2.html">阅读更多</a></p></div><div class="post"><h2 class="post-title"><a href="/2017/01/24/storage-durablity.html">分布式存储系统可靠性-如何估算</a></h2><div class="post-meta">2017-01-24</div><a data-thread-key="/2017/01/24/storage-durablity.html" href="/2017/01/24/storage-durablity.html#comments" class="ds-thread-count"></a><div class="post-content"><h3 id="1-存储系统的可靠性"><a href="#1-存储系统的可靠性" class="headerlink" title="1. 存储系统的可靠性"></a>1. 存储系统的可靠性</h3><p>常规情况下，我们一般使用多幅本技术来提高存储系统的可靠性，不论是结构化数据库存储(典型mysql)、文档型Nosql数据库存储(mongodb)或者是常规的blob存储系统(GFS、Hadoop)等。</p></div><p class="readmore"><a href="/2017/01/24/storage-durablity.html">阅读更多</a></p></div><div class="post"><h2 class="post-title"><a href="/2017/01/22/uploadsystem.html">网易云对象存储服务发布**“直传加速服务”**</a></h2><div class="post-meta">2017-01-22</div><a data-thread-key="/2017/01/22/uploadsystem.html" href="/2017/01/22/uploadsystem.html#comments" class="ds-thread-count"></a><div class="post-content"><h3 id="1-价值创造者的烦恼"><a href="#1-价值创造者的烦恼" class="headerlink" title="1. 价值创造者的烦恼"></a>1. 价值创造者的烦恼</h3><p>打造一款互联网产品时，我们都会非常关注用户体验，当下短视频、线上KTV、线上多媒体互动 等场景越来越多。对于此类产品，我们非常依赖价值创造者，比如美女主播，小视屏制作者，音乐制作人等等。如何为你的价值提供者创造更好的用户体验，这次我们从价值提供者生产并传播价值(上传数据)的用户体验说一说。</p></div><p class="readmore"><a href="/2017/01/22/uploadsystem.html">阅读更多</a></p></div><div class="post"><h2 class="post-title"><a href="/2016/12/28/splitDataWithSalt.html">Split Data With Salt</a></h2><div class="post-meta">2016-12-28</div><a data-thread-key="/2016/12/28/splitDataWithSalt.html" href="/2016/12/28/splitDataWithSalt.html#comments" class="ds-thread-count"></a><div class="post-content"><p>前一阵子团队来一新人，分享了其所从事物联网相关领域的事件经验。其主要核心逻辑是收集处理来自海量的物联网设备上报的一些信息。比如大楼电路、温度、湿度等一些周期性的上报信息，记录这些信息的方式为使用HBase。其中一点是使用Salt 来避免数据分布的不均匀特性。</p>
<pre><code>| EntityID | TimeStamp | Temp | Current| humidity|</div><p class="readmore"><a href="/2016/12/28/splitDataWithSalt.html">阅读更多</a></p></div><div class="post"><h2 class="post-title"><a href="/2016/12/14/nos-from-0-1.html">“网易云存储服务”从0到1发展之路</a></h2><div class="post-meta">2016-12-14</div><a data-thread-key="/2016/12/14/nos-from-0-1.html" href="/2016/12/14/nos-from-0-1.html#comments" class="ds-thread-count"></a><div class="post-content"><h3 id="1-从0诞生"><a href="#1-从0诞生" class="headerlink" title="1. 从0诞生"></a>1. 从0诞生</h3><p> 2012年云计算在业界已经风起云涌，云计算为企业带来的价值为世界所认同，按需提供的计算、存储、网络等底层IAAS基础设施、以及丰富的PaaS和SaaS服务为IT公司产品的开发、运营以及快速可持续发展提供了强有力的支撑。云计算犹如肥沃的土壤，让产品和idea犹如雨后春笋般在晴朗的空气和阳光中快速成长，势如破竹。</p></div><p class="readmore"><a href="/2016/12/14/nos-from-0-1.html">阅读更多</a></p></div><nav class="page-navigator"><span class="page-number current">1</span><a class="page-number" href="/page/2/">2</a><a class="page-number" href="/page/3/">3</a><a class="page-number" href="/page/4/">4</a><a class="extend next" rel="next" href="/page/2/">下一页</a></nav></div></div><div class="pure-u-1-4 hidden_mid_and_down"><div id="sidebar"><div class="widget"><form action="//www.google.com/search" method="get" accept-charset="utf-8" target="_blank" class="search-form"><input type="text" name="q" maxlength="20" placeholder="Search"/><input type="hidden" name="sitesearch" value="https://work-jlsun.github.io"/></form></div><div class="widget"><form action="//www.baidu.com/baidu" method="get" accept-charset="utf-8" target="_blank" class="search-form"><input type="search" name="word" maxlength="20" placeholder="Search"/><input type="hidden" name="si" value="https://work-jlsun.github.io"/><input name="tn" type="hidden" value="bds"/><input name="cl" type="hidden" value="3"/><input name="ct" type="hidden" value="2097152"/><input name="s" type="hidden" value="on"/></form></div><div class="widget"><div class="widget-title"><i class="fa fa-folder-o"> 分类</i></div><ul class="category-list"><li class="category-list-item"><a class="category-list-link" href="/categories/code/">code</a></li><li class="category-list-item"><a class="category-list-link" href="/categories/tech/">tech</a></li><li class="category-list-item"><a class="category-list-link" href="/categories/读书/">读书</a></li></ul></div><div class="widget"><div class="widget-title"><i class="fa fa-star-o"> 标签</i></div><div class="tagcloud"><a href="/tags/HTTP/" style="font-size: 15px;">HTTP</a> <a href="/tags/golang/" style="font-size: 15px;">golang</a> <a href="/tags/performance/" style="font-size: 15px;">performance</a> <a href="/tags/storage/" style="font-size: 15px;">storage</a> <a href="/tags/consistency/" style="font-size: 15px;">consistency</a> <a href="/tags/avaliable/" style="font-size: 15px;">avaliable</a> <a href="/tags/goroutine/" style="font-size: 15px;">goroutine</a> <a href="/tags/runtime/" style="font-size: 15px;">runtime</a> <a href="/tags/交流/" style="font-size: 15px;">交流</a> <a href="/tags/云存储、cdn/" style="font-size: 15px;">云存储、cdn</a> <a href="/tags/云存储/" style="font-size: 15px;">云存储</a> <a href="/tags/CDN/" style="font-size: 15px;">CDN</a> <a href="/tags/浪漫/" style="font-size: 15px;">浪漫</a> <a href="/tags/古典/" style="font-size: 15px;">古典</a> <a href="/tags/算法/" style="font-size: 15px;">算法</a> <a href="/tags/分布式/" style="font-size: 15px;">分布式</a> <a href="/tags/Murphy-s-law/" style="font-size: 15px;">Murphy's law</a> <a href="/tags/nginx/" style="font-size: 15px;">nginx</a> <a href="/tags/Nginx/" style="font-size: 15px;">Nginx</a> <a href="/tags/HTTP-API/" style="font-size: 15px;">HTTP API</a> <a href="/tags/REST/" style="font-size: 15px;">REST</a> <a href="/tags/GC/" style="font-size: 15px;">GC</a> <a href="/tags/Pragraming-Languages/" style="font-size: 15px;">Pragraming Languages</a> <a href="/tags/阅读/" style="font-size: 15px;">阅读</a> <a href="/tags/从优秀到卓越/" style="font-size: 15px;">从优秀到卓越</a> <a href="/tags/S3/" style="font-size: 15px;">S3</a> <a href="/tags/Tunning/" style="font-size: 15px;">Tunning</a> <a href="/tags/性能/" style="font-size: 15px;">性能</a> <a href="/tags/Life/" style="font-size: 15px;">Life</a> <a href="/tags/code/" style="font-size: 15px;">code</a> <a href="/tags/架构/" style="font-size: 15px;">架构</a> <a href="/tags/存储/" style="font-size: 15px;">存储</a> <a href="/tags/分布式、存储/" style="font-size: 15px;">分布式、存储</a> <a href="/tags/分布式、存储、可靠性/" style="font-size: 15px;">分布式、存储、可靠性</a></div></div><div class="widget"><div class="widget-title"><i class="fa fa-file-o"> 最新文章</i></div><ul class="post-list"><li class="post-list-item"><a class="post-list-link" href="/2017/04/18/copyset-replication.html">副本放置&Copyset Replication</a></li><li class="post-list-item"><a class="post-list-link" href="/2017/03/28/a-finding-in-perftest.html">数据存储中Zipf分布</a></li><li class="post-list-item"><a class="post-list-link" href="/2017/03/24/a-hdd-disk-test.html">硬盘性能简测</a></li><li class="post-list-item"><a class="post-list-link" href="/2017/03/17/storage-physical-topoloy.html">大规模存储物理架构</a></li><li class="post-list-item"><a class="post-list-link" href="/2017/02/19/storage-durablity-design-pattern.html">分布式存储系统可靠性-设计模式</a></li><li class="post-list-item"><a class="post-list-link" href="/2017/02/18/storage-durablity-2.html">分布式存储系统可靠性-系统估算示例</a></li><li class="post-list-item"><a class="post-list-link" href="/2017/01/24/storage-durablity.html">分布式存储系统可靠性-如何估算</a></li><li class="post-list-item"><a class="post-list-link" href="/2017/01/22/uploadsystem.html">网易云对象存储服务发布**“直传加速服务”**</a></li><li class="post-list-item"><a class="post-list-link" href="/2016/12/28/splitDataWithSalt.html">Split Data With Salt</a></li><li class="post-list-item"><a class="post-list-link" href="/2016/12/14/nos-from-0-1.html">“网易云存储服务”从0到1发展之路</a></li></ul></div><div class="widget"><div class="comments-title"><i class="fa fa-comment-o"> 最近评论</i></div><div data-num-items="5" data-show-avatars="0" data-show-time="1" data-show-admin="0" data-excerpt-length="32" data-show-title="1" class="ds-recent-comments"></div></div><div class="widget"><div class="widget-title"><i class="fa fa-external-link"> 友情链接</i></div><ul></ul><a href="http://andremouche.github.io" title="Shirly-Blog" target="_blank">Shirly-Blog</a><ul></ul><a href="http://michael-j.net/" title="michael-j" target="_blank">michael-j</a></div></div></div><div class="pure-u-1 pure-u-md-3-4"><div id="footer">© <a href="/." rel="nofollow">Tom Talk.</a> Powered by<a rel="nofollow" target="_blank" href="https://hexo.io"> Hexo.</a><a rel="nofollow" target="_blank" href="https://github.com/tufu9441/maupassant-hexo"> Theme</a> by<a rel="nofollow" target="_blank" href="https://github.com/pagecho"> Cho.</a></div></div></div><a id="rocket" href="#top" class="show"></a><script type="text/javascript" src="/js/totop.js?v=0.0.0" async></script><script type="text/javascript" src="//cdn.bootcss.com/fancybox/2.1.5/jquery.fancybox.pack.js" async></script><script type="text/javascript" src="/js/fancybox.js?v=0.0.0" async></script><link rel="stylesheet" type="text/css" href="/css/jquery.fancybox.css?v=0.0.0"><script>var duoshuoQuery = {short_name:'work-jlsun'};
(function() {
var ds = document.createElement('script');
ds.type = 'text/javascript';ds.async = true;
ds.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') + '//static.duoshuo.com/embed.js';
ds.charset = 'UTF-8';
(document.getElementsByTagName('head')[0]
|| document.getElementsByTagName('body')[0]).appendChild(ds);
})();
</script><script type="text/javascript" src="/js/codeblock-resizer.js?v=0.0.0"></script><script type="text/javascript" src="/js/smartresize.js?v=0.0.0"></script></div></body></html>