r*****d
发帖数: 346 | 1 请问大家：有什么好方法好资料帮助学习Pig Latin? Pig Latin我完全是新手。我在
amazon上找了下，好像没有书评特别好的。
谢谢！ |
D**u
发帖数: 288 | 2 I am currently learning Pig too, looks like more flexible than Hive.
This is the best book so far, Programming Pig.
Try this online version, pretty helpful.
http://chimera.labs.oreilly.com/books/1234000001811/index.html
The author also provided a cheat sheet, very nice too.
http://mortar-public-site-content.s3-website-us-east-1.amazonaw |
D**u
发帖数: 288 | 3 The whole chapter related to Pig from "Hadoop in Action" is open too, very
nice
http://www.manning.com/lam/SampleCh10.pdf |
D**u
发帖数: 288 | 4 That's more than enough I think. |
r*****d
发帖数: 346 | 5 Morale lifted! Thank you Dinu! |
c***z
发帖数: 6348 | 6 You need the following things:
1. An editor, I use sublime2, the cloudera package uses Gedit
2. A cluster with Pig installed at the edge nodes, you can use the VM in the
cloudera package
3. A file transfer to move Pig code from local drive (if you edit locally)
to edge node, I use Winscp, the cloudera package uses Hue
4. A way to run the code at edge node, I use putty, the cloudera package
uses Hue
My work flow: write Pig code locally using sublime2, upload code to edge
node using winscp, run code at edge node using putty.
【在 r*****d 的大作中提到】
: 请问大家：有什么好方法好资料帮助学习Pig Latin? Pig Latin我完全是新手。我在
: amazon上找了下，好像没有书评特别好的。
: 谢谢！
|
r*****d
发帖数: 346 | 7 多谢多谢。就知道你会回复 ：）
the
【在 c***z 的大作中提到】
: You need the following things:
: 1. An editor, I use sublime2, the cloudera package uses Gedit
: 2. A cluster with Pig installed at the edge nodes, you can use the VM in the
: cloudera package
: 3. A file transfer to move Pig code from local drive (if you edit locally)
: to edge node, I use Winscp, the cloudera package uses Hue
: 4. A way to run the code at edge node, I use putty, the cloudera package
: uses Hue
: My work flow: write Pig code locally using sublime2, upload code to edge
: node using winscp, run code at edge node using putty.
|
c***z
发帖数: 6348 | 8 Fishing post? :P
Thanks for the baozi.
Additionally, an job tracker might help, the cloudera package uses Hue, and
I use the company setting. |
c***z
发帖数: 6348 | 9 Also, my work flow using Scala Scalding (i.e. Scala on Hadoop):
1. edit and compile in Intelij, or other IDE, or edit in any text editor and
compile in a terminal (CMD for windows)
setting up Intelij is complicated and out of my scope
2. upload the jar file to edge node, I use winscp
optionally, upload the code to github for version control
3. run the jar file at edge node using putty, with specific input path,
output path and other argument, I save them to an .sh file for reuse (you
can save to .txt file and then copy and paste)
the command to run the jar is something like
hadoop jar myjar.jar packagename.functionname --input "myinputpath/part*" --
output "myoutputpath" --hdfs
4. to get the output to a text file, use something like
hadoop fs -cat "myoutputpath/part*" > myresult.tsv
(I prefer .tsv over .csv because comma can appear in numbers like 133,010
and mess up things) |
c***z
发帖数: 6348 | 10 The difference between the pig method and the scala method is that scala is
not installed at
the edge node, and we compile a jar to run at the edge node
http://nosql.mypopescu.com/post/18004413595/an-introduction-to- |
|
|
s*********e
发帖数: 1051 | |
g**********l
发帖数: 214 | 12 请问 what is the "real" advantage of scalding over pig?
想找学scalding的动力, for analyst/data scientist。
i see one "real" advantage of pig over scalding is the utilization of
hcatalog.
公司用很多hive做dataware housing 的话，pig can access all those tables's
metadata directly. no need to parse anything. this is especially helpful
when you have hundreds or even thousands of columns.
pig can also write to hive via hcatalog so other coworkers (who uses mostly
hive) can use my result easily.
另外就是pig's UDF libraries, like datafu. 还有就是scripting language (like
python) support for UDF.
如果大牛们能给一些工作中的实例to show advantage of scalding over pig, 那就更
好了。 |
c***z
发帖数: 6348 | |
D**u
发帖数: 288 | 14 nice， share workflow最受用了。
顺便分享一下，我现在学习用Notepad++写Python的Pig UDF，然后用Jython 在 Linux
Putty里compile。
还有一种很流行的方式是用Maven+Eclipse 写java的 Pig UDF，稍微学习了一下，还没
有领悟精要。
and
【在 c***z 的大作中提到】
: Also, my work flow using Scala Scalding (i.e. Scala on Hadoop):
: 1. edit and compile in Intelij, or other IDE, or edit in any text editor and
: compile in a terminal (CMD for windows)
: setting up Intelij is complicated and out of my scope
: 2. upload the jar file to edge node, I use winscp
: optionally, upload the code to github for version control
: 3. run the jar file at edge node using putty, with specific input path,
: output path and other argument, I save them to an .sh file for reuse (you
: can save to .txt file and then copy and paste)
: the command to run the jar is something like
|
c***z
发帖数: 6348 | 15 学习了，多谢！
Linux
【在 D**u 的大作中提到】
: nice， share workflow最受用了。
: 顺便分享一下，我现在学习用Notepad++写Python的Pig UDF，然后用Jython 在 Linux
: Putty里compile。
: 还有一种很流行的方式是用Maven+Eclipse 写java的 Pig UDF，稍微学习了一下，还没
: 有领悟精要。
:
: and
|
h***x
发帖数: 586 | 16 Notepad++ is also a good IDE for SAS programming. I like to use it to edit
SAS codes and run in batch mode in PC windows, similar to the way in Unix.
Linux
【在 D**u 的大作中提到】
: nice， share workflow最受用了。
: 顺便分享一下，我现在学习用Notepad++写Python的Pig UDF，然后用Jython 在 Linux
: Putty里compile。
: 还有一种很流行的方式是用Maven+Eclipse 写java的 Pig UDF，稍微学习了一下，还没
: 有领悟精要。
:
: and
|
r*****d
发帖数: 346 | 17 """
This is the best book so far, Programming Pig.
Try this online version, pretty helpful.
http://chimera.labs.oreilly.com/books/1234000001811/index.html
"""
正在读，确实是一本好书！
【在 D**u 的大作中提到】
: I am currently learning Pig too, looks like more flexible than Hive.
: This is the best book so far, Programming Pig.
: Try this online version, pretty helpful.
: http://chimera.labs.oreilly.com/books/1234000001811/index.html
: The author also provided a cheat sheet, very nice too.
: http://mortar-public-site-content.s3-website-us-east-1.amazonaw
|
s******0
发帖数: 1269 | 18 强贴留名，for future reference |
