hadoop - How do I read a block from a specific replica in HDFS? -

-

i need write program pulling out blocks in hdfs from specific data node given file. not run within mapreduce job, pro grammatically gathering statistics on cluster.

so far, given file, have been able extract block locations associated file. each block there multiple replicas , need access blocks lying on replica. understand these remote read operations.

i have done following:

configuration conf = new configuration(); filesystem fs = filesystem.get(conf); path infile = new path(args[0]); filestatus status = fs.getfilestatus(infile); blocklocation[] locs = fs.getfileblocklocations(status, 0, status.getlen()); string[] hosts = locs[0].gethosts ();

could tell me how can fetch block pointed locs[0] hosts[0]?


Comments

Post a Comment

Popular posts from this blog

python - Subclassed QStyledItemDelegate ignores Stylesheet -

-
i'm trying subclass qstyleditemdelegate remove focus rectangle in qcombobox . even though i'm calling base implementation of paint function , nothing else, result different. looks if parts of style-sheet influence bounding box of item taken consideration. class pstyleditemdelegate(qstyleditemdelegate): def __init__(self, *args, **kwds): super(pstyleditemdelegate, self).__init__(*args, **kwds) def paint(self, *args, **kwargs): qstyleditemdelegate.paint(*args, **kwargs) what have paint unmodified qstyleditemdelegate ? as suggested tried replacing pyside pyqt4 , works, appears bug. update pyside 1.1.2 1.2.1 result same. unfortunately switch breaks other parts of code, if there no other suggestions i'm going accept answer. edit bug tracked here
Read more

java - HttpClient 3.1 Connection pooling vs HttpClient 4.3.2 -

-
i have used httpclient3.1(java) connection pooling below : import org.apache.commons.httpclient.httpclient; import org.apache.commons.httpclient.multithreadedhttpconnectionmanager; import org.apache.commons.httpclient.params.httpclientparams; import org.apache.commons.httpclient.params.httpmethodparams; import org.apache.log4j.logger; public class httpclientmanager { private static logger logger = logger.getlogger(httpclientmanager.class); private static multithreadedhttpconnectionmanager connectionmanager = new multithreadedhttpconnectionmanager(); private static httpclientparams clientparams = new httpclientparams(); static { try { clientparams.makelenient(); clientparams.setauthenticationpreemptive(false); clientparams.setparameter(httpmethodparams.so_timeout, new integer(30000)); // set so_timeout clientparams.setparameter(httpmethodparams.head_body_check_timeout, new integer(3000)); // head responses
Read more

SQL: Divide the sum of values in one table with the count of rows in another -

-
is possible to, in single query, sum bunch of values in 1 table, , divide count of rows in another? there common id/key in both tables... fyi, win points least info provided in question. answer it's possible...here psuedo code since i'm guessing @ tables , columns select summing / counting whynot (select id, sum(whatever) summing where_ever ) inner join (select id, count(1) counting what_ever)b on a.id = b.id question = 'vague' group a.id
Read more